The listicle generator, or how I automatically annoyed you

Dominik Bärlocher

11.12.2018

Translation: machine translated

Yesterday, we published a completely stupid article. And why? Because it shows that journalism can be done by machines and still be effective. I generated a listicle semi-automatically.

Do you think listicles are silly? I think so too. You thought my listicle on cables yesterday was super badly done? That's fine. You've got it all figured out. Listicles are silly, and I'll show you just how silly. I can even generate them pre-formatted on a web application, semi-automatically. After a few hours of development, I managed to save about an hour per listicle.

I'll show you how I developed the web app, what it does and how it could be improved.

What makes listicles successful?

These kinds of articles represent a definite interest for online portals wishing to attract visitors. Indeed, they have some convincing arguments.

• A defined number of articles, dangling a form of exclusivity → "The 14 best cables"
• A trigger that calls out → "Number 9 will impress you"

With this text, I arouse your curiosity while stirring up emotions. You probably think: "I'm sure number 9 is worthless" and click on it. You want to know what it is, and if it's as bad as you imagine. Portals like Buzzfeed have reached millions of internet users with headlines like "The 14 best cables: Number 9 will impress you". It doesn't matter how loyal that audience is. Every visitor goes to the page, looks at the image and thinks it's complete rubbish, but they've seen the ad, created an impression, and therefore generated revenue.

Journalists find listicles even dumber than readers. Indeed, what many forget is that someone has to write these turnips. Management is divided on the value of listicles. It's a subject that comes up in discussions about the search engine optimised (SEO) word 'shopping guide'. As they always work in the same way, I've automated them. I don't want to spend hours formatting my articles, however the idea of spending a lot of time programming doesn't scare me.

Describing a listicle

In order to create a project such as my "Black Magic Listicle Generator", you need to know the individual components of a listicle.

It's important to know that listicles don't read like a book. Beyond the numbers and the attractive title, no one is interested in the texts. The content is therefore of little importance. On the other hand, it is essential for pages to be well referenced by Google's algorithms.

A listicle is made up of the following elements, specific to digitec in my example:

• The title of the article
• The lead: this is the text below the title. Its font is slightly larger than the rest.
• The introduction: usually one or two sentences located between the lead and the list, because Google's indexing punishes us if we go straight from the lead to the subtitle.

*The heading: The part that announces the item numbers and contains a bit of text.

• A list of products
• Free text: this must be relevant in terms of SEO, but doesn't necessarily have any specific content or meaning. Ideal for words that boost SEO.

I've used "SEO", or "search engine optimisation" in good French, many times before. Behind this term hides a religion rather than a science, which is hugely profitable for Google. SEO is all about getting content better indexed by Google. Nobody knows the exact criteria, but certain common practices have been established. For example, a headline 3 cannot exceed a headline 2. Synonyms for individual words - smartphones, telephones, natel, mobile - broaden the search field, even though Google now does this itself. And there are others. In every media company, there's always someone who thinks they can get millions of unique, organic visits straight away thanks to SEO. So everything has to be optimised immediately, no matter how readable the text.

At least, that's my opinion.

A listicle, then, is something that machines like, but no one reads. If we follow this principle, why shouldn't a machine write it itself?

JQuery to the rescue

I occasionally like to program web apps that perform a specific task. They have one task, don't support any error handling and work within a very limited framework. I'm probably the worst coder in the world, but I still manage to create a listicle generator. If I can do it, so can you.

Here's what you need:

" A text editor like Brackets

• A ready-to-use CSS stylesheet such as Materialize
• Some knowledge of HTML
• Some knowledge of JQuery
• The ability to copy and paste

The project we're creating looks sophisticated, but it's work done in a hurry. Yep, behind the scenes, everything is created a bit on the hoof. And that's the idea: we're not going to get hung up on listicles. On the contrary, we want to delegate as many tasks as possible to the machine.

Create a folder structure like this:

root/
├── css/
│ └── styles.css
├── img/
│ ├── logo.png
│ └── favicon.png
└── index.html

Small explanation:

• css/styles.css: special cases that concern the design of the page. You use this file to change colours or something like that, if necessary.
• img/logo.png: each page needs a logo (optional)
• img/favicon.png: each page needs a favicon (optional)
• index.html: this is where the main work is done. As we want to do everything quickly, our code won't be super elegant.

The HTML skeleton

For this project, we only need the JQuery JavaScript library, but I want to create a skeleton from multiple worlds that covers every use case and programming language I'll need for web apps. Here it is:

The experts can already see that it's quick and dirty, since I'm incorporating far too many elements that I don't need for a JQuery project. I've never claimed to be a good coder.... And I don't want to waste time. I'd rather go and have a beer than spend hours on it.

At the end of the day, I want to get text that follows this pattern:

1. Title
2. Lead
3. Introduction
4. Title
5. Product
6. Unimportant filler phrase that serves SEO purposes

The abstraction continues

We can divide up the above elements even further to be able to achieve the most versatile webform possible and get to the result quickly.

The title

A headline always has a similar, if not identical, format: "The $number $productittPluralType. Number $number2 will impress you"

We can ignore $number and $number2 for now. They don't need any human input. Here's another listicle secret: the famous number 9 is absolutely useless. In fact, I still don't know what the number 9 in yesterday's listicle is. I let the computer calculate the total number of digits and mark one at random as a special digit.

But $produitTypePluriel is a little more complex, because French grammar isn't the easiest. I need to give the singular and plural of the product types to the machine. In the title, I need the plural, but I'll use the singular in the intertitle.

So here's what the HTML code for the headline looks like:

This/this/these

I need to provide some grammar rules to the machine for the heading. Indeed, "this cable" still appears in the current version. It is, of course, generated automatically. Here, the variables are $productTypeThis and $productType.

The lead and the introduction

I keep these two elements as free text. It makes the most sense in SEO terms. And two, three sentences are quickly written.

The heading

The "this cable" heading consists of the following elements:

var articleZwiti = productTypeThis + " " + productType

This is followed by an automatically generated list and a bit of formatting.

The product

We insert the products manually into our backend, as usual. We need the product number, which is embedded in each product URL, which is handy.

Since we can insert as many products as we like, I'll then have to work with an array. So I create a text box that accepts one link to the digitec shop per row and work with it.

The useless fake text

Since no one reads the fake text, I can incorporate words that will improve SEO. Again, the text box consists of one sentence per line. The first line is then allocated to the first product, the second to the second product, and so on.

I have also added a button below, as my generator can work with these indications and create a complete listicle.
The listicle is generated in a text box called #listicleOutput.

JQuery to the rescue

At the moment, the page is certainly pretty, but it doesn't yet perform any functions. As this function isn't particularly complex, we can work with JQuery. This JavaScript library is very robust and versatile, perfect for projects like our hack.

The script part of the index.html page allows us to achieve lots of things with very few lines of code. But first we need to go over the terms defined at the very top of my article. We do this using variables:

var productNumbers = $('#productNumbers').val()
var uselessSentences = $('#uselessSentences').val()
var productType = $('#productType').val()
var productTypePlural = $('#productTypePlural').val()
var productTypeThis = $('#productTypeThis').val()
var articleLead = $('#articleLead').val()
var articleIntro = $('#articleIntro').val()

Now the digitec-specific aspects come into play for the first time. In the backend, we work with Markdown. This means we can format using single characters before each line. These are then taken over by the website. It's very practical. Marketing managers and copywriters don't have to format articles 50,000 times.

var zwitiPrefix = "## ";
var productPrefix = "[[product: ";
var productSuffix = "]]";
var doubleBreak = "\r\r";

Small explanation: when I create a heading, this is what it looks like in the backend:

The heading

An inserted product looks like this:

[product: 8287505]

Two line breaks need to be inserted between the individual formatting. I define them as a variable with var doubleBreak. With these formattings and the directions from the webform above, I can create my listicle.

Two arrays with quick data

I'm feeling lazy. I just want to embed links in my webform, because that's also how I find the products in my poor listicle. Editing product numbers manually while staying focused is asking too much of us. We're going to do it automatically.

var urls = productNumbers.split(/\s+/)
console.log(urls)
var mv = $.map(urls, function (val, i) {
if (val != "") {
var ju = val.split('?')[0]
pat = /-|/$/
rv = pat.exec(ju)
if (rv !== null) {
console.log("Found id: " + rv[1])
return rv[1]
} else {
console.log(" Not matching html ")
}
}
})

I create an array with the URLs. Generally, the product number is on the far right of the URL.

https://www.digitec.ch/fr/s1/product/caterpillar-s61-520-64gb-dual-sim-16mp-schwarz-silber-mobiltelefon-8287505

But when I get to the product page I've chosen via the website filter, the page creates a suffix that matches the filter at the back of the URL.

https://www.digitec.ch/fr/s1/product/caterpillar-s61-520-64gb-dual-sim-16mp-schwarz-silber-mobiltelefon-8287505?tagIds=82

This explains the .split and the question mark. Anything after the question mark is rejected. I then choose a series of numbers from 5 to 8 and add them to the array. Thanks to StackOverflow, I can even integrate error handling.

The second array is quickly done too, as the texts are a bit easier.

var lines = $('#uselessSentences').val().split(/\n/);
var texts = [];
for (var i = 0; i < lines.length; i++) {
// only push this line if it contains a non-whitespace character.
if (/\S/.test(lines[i])) {
texts.push($.trim(lines[i]));
}
}

I created this using StackOverflow. Basically, I'm doing nothing more than integrating the phrases that serve SEO purposes into a second array.

We build an article

We have everything we need to create our article. In JQuery, it's easy with .each:

$.each(mv, function (index, value) {
var articleZwiti = productTypeThis + " " + productType
$('#listicleOutput').append(zwitiPrefix + (index + 1) + ": " + articleZwiti + doubleBreak + productPrefix + value + productSuffix + doubleBreak + texts[index] + doubleBreak);
})

I take the variables at the top, quickly define what a heading consists of, and off I go:

• In the #listicleOutput field
• Attached to the content (empty so far)
• The two hashtags, so that the system creates a heading
• Next, the places in the arrays where the product number can be found. Please note: arrays always start with a zero, whereas editors start a list with 1. This explains the +1 at the end.
• Two points
• The heading
• Two blank lines
• The two square brackets and the word "product:"
• The product number
• Close the two square brackets
• Two blank lines
• The corresponding sentence in the array texts
• Two blank lines

each does this for each entry in the link list. It also sorts the sentence from the two arrays.

Clicking the Black Magic button does everything automatically, and the finished listicle outputs to #listicleOutput.

The title

A listicle has a title, which can be automated. All you need is the number of entries in the array (+1), then a number taken at random from the entries in the array.

var randomIndex = Math.floor(Math.random() * texts.length)+1;

var articleTitle = "Die besten " + texts.length + " " + productTypePlural + ". Nummer " + randomIndex + " wird dich erstaunen";

The variable randomIndex chooses a number at random from the array and then creates "Number 9 will impress you". With the .length attribute, I'm telling JQuery that it's the length of the array that's important, not its contents.

We move this construct to the beginning of the listicle:

$('#listicleOutput').prepend(articleTitle + doubleBreak + articleLead + doubleBreak + articleIntro + doubleBreak)

That's it, the listicle is complete and can be copied identically into our backend. A listicle can therefore be tweaked in a matter of seconds.

You can also enhance this template as you see fit. The multitude of products from digitec and Galaxus? No problem. Lists of styles in which an outfit needs to be illustrated? No problem. Human error is minimised. The content is certainly optimised for the CEO, but it's not relevant.

The other elements

We're not quite done yet. As we want to do as little as possible, we can automate as many tasks as possible at the click of a button.

Copy listicle

If we don't copy the listicle automatically, we have to highlight everything in the #listicleOutput field, right-click and then copy or hit CTRL/⌘+C. This again represents a risk of human error. So:

$("#listicleOutput").select();
document.execCommand('copy');

This copies the entire content to the clipboard. We can save ourselves this step, as well as the manual search for a corresponding image at the top. In fact, we've already specified at the top the search term we'd type into Google, in $productType.

var searchURL = "https://www.google.ch/search?q=" + productType + "&tbs=sur:fc,isz:l,imgo:1&tbm=isch"
window.open(searchURL, '_blank')

Here it is, a Google image search window opens at the touch of a button. The suffix at the end of the URL does nothing more than indicate the following filters in Google's image search in a standardised way:

• Size: large
• Licence: royalty-free
• Show image size

In fact, no listicle is complete without a royalty-free Stock photo at the top.

It's your turn: carry on.

I've grown quite fond of this project. With a little extra effort, the generator can do so much more. I wondered if I could submit other aspects to it such as "the best", "the most beautiful" so that useless but important phrases for SEO could also be generated or taken from an array.

That's it, I'm done. I'm now turning my attention to a project that requires research and passion. Journalism, for example.

Updated 11.12.2018 // 10:55am GitHub

As you asked us in a comment, here's a GitHub directory that contains the project. Show me what cool things you've created with it.

Dominik Bärlocher

Senior Editor

dominik.baerlocher@digitecgalaxus.ch

Journalist. Author. Hacker. A storyteller searching for boundaries, secrets and taboos – putting the world to paper. Not because I can but because I can’t not.

Follow

These articles might also interest you