Some tips for creating a (good) CTF

  • Warning: this article is the sole responsibility of its author. OSINT-FR condemns any malicious use of the methodologies described in this guide.

The OSINT CTF (Capture The Flag) are fun challenges that allow to highlight and improve the Open Source Intelligence (OSINT) techniques of the participants and of those who read the feedback (even if, of course, nothing beats practice). Concretely, these CTFs are guided by questions, where each correct answer unlocks the next. The TraceLabs challenges, which aim to help find missing persons, are an exception.

Many high quality CTFs have been created lately: Hexa OSINT, created by Sopra Steria during the Fabrique Défense, Stranger Case by ESNA, the UYBHYS CTF, prepared by members of OSINT-FR, the Bleuet de France and the Hunt by AEGE, and so on. For my part, I had the opportunity to participate in the creation of the OSINT 317 challenge during the 2021 and 2022 editions of the European Cyber Cup, which took place during the International Cybersecurity Forum (ICF). After (many) hours of working on these two projects, with very positive feedback, I propose to share a few tips on what it takes to create a good BTC, hoping to spark vocations and/or facilitate the work of organizers.

1. Be consistent

Consistency is, in my opinion, the main element of a good CTF, because everything follows from it. Indeed, the goal of the challenge is to create a fictional investigation and, in any investigation, the investigator is confronted with people who act according to a certain logic.

The first step is to write a scenario with a general plot and an exhaustive description of the characters, making sure that they keep a similar digital behavior on the different platforms where they have avatars. For this, it is recommended to keep an Excel document with all the elements related to the avatars (date of birth, accounts on RS, passwords, etc.).

The scenario is then broken down into questions, taking into account the chronology: the participants must be guided step by step (one of the limitations of classic CTFs, since a real survey is marked by uncertainty). To do this, we can use a collaborative solution like Miro. In the example below, it is important to understand that solving question 1) unlocks question 2) of the “Ian” trail and the first question of the “Debbie” trail (which allows the participants to work on several questions in parallel).

It goes without saying that the resolution of each question must be consistent. If two avatars communicate discreetly, shorthand can be used. However, using this technique to hide the date of birth of an avatar would not be logical (and therefore almost impossible for the participants to find).

2. To encourage reflection and analysis

The fear when creating a challenge that is supposed to last 24 hours is that it will be completed in only a few hours. So you need to create questions that are complex enough to be feasible while still getting participants to think and hypothesize about their research. But then, how do you create good questions?

The method used for the 317 challenges is to keep a table of all the techniques you would like to use to solve the puzzles. Some of the more classic ones are: bouncing from a pseudonym, using archives, geolocation, etc. This implies keeping up to date with the techniques in order to remain innovative and creative. If the scenario is well defined and each character has well-fed accounts on social networks, blogs, etc., the only thing left to do is to ask yourself: what information do you want to uncover and by what technique?

Finally, the questions must be tested by a team placed in the same conditions as the participants. This step allows to realize the difficulty but also the possible mistakes, either in the formulation of the questions or even in the answers. Ideally, this testing phase should be completed one week before the challenge.

3. Create a maximum of noise

The major difficulty of a real investigation lies in the large number of false leads and the almost infinite number of research hypotheses: an individual can be totally invisible on the Web (but you still have to be sure!) or have dozens of accounts, be active on forums, blogs, etc. And in this case, you have to extract all the interesting elements. And, in this case, it is necessary to extract all the interesting elements. Because, we can’t repeat it enough, finding an account/blog is useless: what is needed is to analyze it thoroughly and collect all the elements.

Applied to a CTF, this means that it is necessary to create noise, false leads. You have to “give life” to the characters, by feeding their accounts, so that not all articles, tweets, post-blogs, etc. are published on the same day. If one of the avatars has an Ask FM account and there is only one publication on it, the clue is too quickly collected. Similarly, if you want to hide a clue on your Facebook account, it is necessary to publish many posts in order to force the participants to read everything, as you would in a real investigation. These posts must be part of the overall coherence of the scenario, especially if the characters interact with each other. Again, this requires organization to define “who” publishes what and when, especially for important messages. If a team is in charge of organizing the CTF, I recommend that a character’s posts always be made by the same person, in order to keep the tone, the words, etc. So, you have to define the roles from the beginning of the CTF and stick to them. Each person must publish regularly (according to the profile of the character), in order to hide the key messages in the mass of publications.

We also touch on the limit of the exercise: creating a scenario and characters from scratch cuts us off from all legal documents (business registers, databases, etc.). You can always tinker (for example, ask the name of the founder of a restaurant where one of the avatars had lunch), but this has its limits.

4. Confronting your ideas

Another tip for making a good BTC: constantly compare ideas. Just like a writers’ room for writing a series, the creation of a CTF is fed by the ideas of others. Sometimes one person comes up with an idea, which is enriched by another, then amended by a third. Regular meetings allow not only to follow the progress of the global project, but also to offer opportunities for this confrontation of ideas. Let’s remember that ideas should not emerge simply because the scriptwriters meet, but that everyone should come up with ideas, otherwise it could turn into a meeting.

In the end, without a good methodology, a lot of time and energy can be lost. That’s why I wanted to share these tips. Building a CTF requires time and organization, and a good level of OSINT so that the challenge is there for the participants.

If you have any other ideas or would like to discuss them, please feel free to comment or contact me.