Tanks, Trains, and TikToks
Tanks, Trains, and TikToks
How you, too, can track massive Slavic armies from your couch
Today’s post includes:
How to use some basic code to find updates on the Russian military buildup near Ukraine
That’s pretty much it
But first of all…
Need to convert geo coordinates into useful location information?
This week's edition is sponsored by OpenCage who operate a highly available, simple to use, worldwide, geocoding API based on open data. Try the API now on the OpenCage demo page.
You’ll notice that today’s edition of Line of Actual Control includes, for the first time ever, a small ad! You’re probably thinking: has the ol’ bastard finally sold out? Is this the beginning of the end for LOAC? Does he desperately need to fund a gambling addiction? No, no, and no. I’ve partnered with OpenCage for the simple reason that they provide an incredibly useful, entirely open-source service to help people who are passionate about geo data make sense of that information.1
Now onto the main event…
In the pre-digital age, there was a weird phenomenon in which a war would continue after a peace treaty was signed simply because the combatants hadn’t heard that the war was over. This is how we got, for instance, the Battle of New Orleans.
In the digital age, that phenomenon has reversed. A quick look on social media shows the earliest indications of a conflict before the war has even started. In this phenomenon, there exists an opportunity, however slim, for belligerents to improve their understanding of the other side and potentially defuse tensions before things turn hot.
But what does that actually look like? In practice, it looks like TikTok-ers across Russia filming hundreds of trains loaded with tanks, trucks, and other materiel heading to the Ukrainian border.
There are some pros on social media who collect and post these videos - Igor Girkin is one:
And while I gave it a shot, I quickly realized I wasn’t very good at it (probably because I speak no Russian whatsoever). So I did the next best thing - I looked for a way for code to find these videos for me.
After casting around a bit, I stumbled upon a GitHub repo for a TikTok scraper. One of the cool features of this code is that it allows you to both download TikTok videos and the videos’ metadata. While downloading the videos themselves can be good, downloading more than a few will rapidly use up your computer’s memory (and take a lot longer than you probably want it to). So I concentrated on the latter option: using the scraper to search through TikTok metadata for videos about the Russian troop buildup.
By the way, despite the esoteric name, metadata is just the extraneous information attached to a file, such as the date it was created, the file’s size, any links to the file, and so on.
Anyway, my process has a few moving parts, so let’s dive right in.
First and foremost, I made a list of the hashtags I wanted to find videos about and saved the list as a csv file. Each TikTok video can include one or more hashtags that users can click on to find more videos about that particular topic. Since I was just generally curious about Russian troops moving toward Ukraine, I composed a list of generic Russian military terms:
But you could also compose more topic-specific hashtag lists. For instance, if you want to find TikToks about the trains that carried the equipment, you could include hashtags like #rail, #railroad, #railway, #train (all in Russian of course). Or if you want to surface posts about the Russian navy, you could do #port, #ports, #ship, #shipping, etc. You get the gist.
Next up is where the magic happens. I opened a new terminal window and entered the following command:
tiktok-scraper from-file /Path/to/my/hashtag/file/hashtags.csv 5 -t csv --filepath /Path/to/my/output/folderThere’s a lot going on here, so let’s break it down piece by piece.
If you enter everything correctly (no typos!), you’ll see the scraper’s progress in real time and, when the task has finished, a sweet, 1990s-esque table about the metadata you downloaded for each hashtag will appear.
I checked my destination filepath to make sure all the csvs containing the hashtags’ metadata are there:
And then moved onto the second main part of the technique: concatenating all of the scraped csvs into a single file and sorting the metadata contained in that file by the time each TikTok mentioning the hashtags was posted.
If you’re interested, feel free to read all the comments in the code above for the ins and outs of what the program actually does. If you’re not (and I don’t blame you…), suffice it to say that I’m basically smooshing all the hashtag metadata files into one and sorting it so it shows only the most recent results.
If you open the output master file in Excel or Google Sheets, you’ll see a huge table containing tons of different metadata fields about TikTok videos mentioning the hashtags - the date and time each video was posted, the number of views each video has, information about the music contained in each video, and so on.
But the field I focused on was the webVideoUrl, which is essentially just the web link to each TikTok:
I opened the links in that column and pretty much immediately started seeing interesting TikToks about the Russian military buildup. I started running the scraper roughly once a day in mid-December and while it surfaced maybe one or two results per day then, it has since escalated to six or eight valuable posts per day at time of publication (late January 2022).
The video above, which was one of the first posts the scraper surfaced, shows a Tor surface-to-air missile system being transported by a semi-trailer.
Another TikTok the scraper pulled in was the following video of an S-300 or S-400 air defense system traveling through Vladivostok. These missiles can not only shoot down high-flying aircraft or ballistic missiles but are also traveling from the far eastern edge of Russia. If these missiles ended up near Ukraine, they would have made a journey of thousands of miles across the country - indicating the immense breadth of the Russian mobilization.
Speaking of long journeys, the comments on the TikTok below indicate that the gear on the train is coming from Ussuriysk, another town in far eastern Russia, and is heading to either Ukraine or Belarus (although the consensus from the comments seems to be Belarus).
This technique has also allowed me to keep tabs on trends in the mobilization. For instance, while the first convoys were deployed to areas of Russia near eastern Ukraine, the scraper started picking up videos in mid-January of troops in Belarus, which is where Russian forces have increasingly concentrated. This video shows a line of troops driving through Grodno, Belarus, near the Polish and Lithuanian borders.
I also posted a Twitter thread of more findings from the scraper, if you’d like to peruse it or use it do your own research:
There are, as always, a few limitations. For instance, I’ve noticed some variability in how much metadata gets pulled in during each scrape. Results also vary depending on whether you’re using a VPN or if you’re logged into TikTok itself.
Furthermore, there is room for significant expansion of this technique by folks who are more familiar with the Russian military than I am. Users could build hashtag lists focused on each service branch of the Russian armed forces, the Russian rail and road networks, the Ukrainian response, or really anything else.
From a technical standpoint, you could put the scraper in a cloud service so it runs automatically or download the videos to external or cloud storage if you want to refer to the videos later. Another cool feature could be changing the scraper’s angle of attack. Instead of looking for hashtags, could you search for common music clips used as soundtracks in these videos? Or topical or location-based keywords used by people posting the TikToks?
At any rate, if you use this method or can think of any ways to improve it, I’m all ears. Of course, let’s hope the situation in Eastern Europe doesn’t come to blows, but if it does, this method may provide insight into conditions in the region.
Finally, I wanted to say a major thanks to analyst Tom Bullock for his help on the IDs, code review, editorial contributions, and translations in this piece. Give him a follow for anything Russian military (or TikTok!) related.
You may continue to see ads for services I’m particularly passionate about in the future, with the promise that I will never sell your data or compromise the quality of the newsletter. In the meantime, I’m always eager for feedback on ads or anything else. Feel free to comment on the post, DM me on Twitter, or email me at lineofactualcontrol@protonmail.com. Want to advertise? Give me a shout!
easy way to hide your filepath: https://github.com/henriquebastos/python-decouple/