Stonks? More Like Stinks
Making important financial tools is pretty easy, why isn't everyone doing it?
This is the story of how my online-addled brain misfired some neurons and an objectively dumb and bad idea was created. That idea would bounce around inside my skull much like the DVD logo screensaver on an old TV for the better part of a week before I decided it needed to be removed, the only way I knew how: by making it a reality. This is the story of Stonks.info - Info about Stonks.
The Bad Idea
The weird Stonks meme has been floating around for a while now, old enough now to be a meta-meme in itself. In case you somehow haven’t seen it before, this is essentially the whole thing:
There’s no real deeper meaning to it than just what you see, and that simplicity is probably the reason it’s had such a strong run. That’s how it came to be in my Twitter timeline one day, just a response to some story about Trump doing a bad tweet that tanked the market for the day. By now I can’t even remember what particular story it was about or who made the joke, but it did lodge a thought inside my head: a real-time stonks tracker. When stonks are good, show the stonks going up. When stonks are bad, show the stonks going down.
What Is Stonks?
So, what does “stonks are good” even mean? Honestly, who cares. For most people, the stock market is just an abstraction on top of a mountain of abstractions that reduces the complexity of the whole economy down into a simple yes/no answer to the question, “are we happy about money today?” Even if you’re like most people and don’t have any investments, it’s a way to tell if Financial Things™ are okay or not, and whether that’s something you have to worry about today. So, why make it more complicated than it has to be? Are the stonks good? Nice. I am happy to know that. Uh-oh, the stonks are bad today? That’s concerning, but at least they’re only a little bad, so I am only a little concerned.
From there, what’s the most commonly accepted, probably not accurate, but accurate enough metric by which “The Market” as a whole is typically measured? The famous Dow Jones Industrial Average, of course. What does it actually represent, here in the year of our lord 20whatever? ¯\_(ツ)_/¯ It means that when it goes up, a lot (or most) of the rest of the market does too. And since we don’t actually have a true average of all the market activity for the day, the Dow is about as representative a real-time indicator of the whole damn thing as there exists. And that fits perfectly into our model of extremely non-specific definition of good and bad, the details of which are so unimportant to you that you probably forgot what the question was while you were asking it.
Checkpoint: You know this is still dumb as hell, right?
Oh my god, yes. Did it sound like I was reverse engineering some kind of high-minded rationale behind this? Like I was trying after the fact to make out like I was doing everyone a favor here by obfuscating the “big confusing stock market” for all our “tiny baby brains”?
No, listen. This is idiot shit. For idiots. I’m keeping track of how many people visit the site per day and the lower it is, the less I fear for our collective future.
Getting That Data
For a long time, Yahoo! maintained lot of free public APIs for everything from weather to stocks to fantasy football. You didn’t need any kind of authorization or authentication, they just spit json at you whenever you asked for it. But now that the internet is slowly contracting into just a single massive website presided over by Mark Zuckerberg, they’ve been gradually disappearing. And that’s how the literal first concrete step in this project hit a surprising roadblock: no one will give you an arbitrary amount of real-time stock data for free. The main reason behind this seems to be licensing of the data itself. The NYSE and all its friends have, in a very on-brand way, decided that people using their data for free is no good at all and now restrict it with the same sort of dense terms of service that Apple uses to prevent you from using iTunes to develop nuclear weapons.
There does still exist a way to get the same data from Yahoo!, since they feed into things like all of Apple’s various stock widgets and Alexa’s responses to market-related questions, but it’s unofficial and private, so they could change it without any warning and my precious financial tool could be broken instantly. And if this thing ever breaks, I am not putting any amount of effort into fixing it. It will remain as its own tombstone for all eternity.
This leaves us relying on one of a number of resellers of market data, and I’m pretty sure I’d be (rightly) sent to prison for spending any amount of actual money on this, the dumbest idea I’ve ever had, so we’ll have to stick to those offering a Free Tier. After weeding through a number such platforms, Twelve Data seemed to be the best fit:
- Free API usage
- Simple endpoint for current value of a symbol (DJI)
- Real-time data
- Limited to 8 API requests per minute; 800 API requests per day
My original conception of how the actual site would function was that it’d be a simple static site using basic javascript making a request to the Yahoo! API (RIP) when the page loads, and maybe every 60 seconds after that. There’d be no server-side code to worry about or persistent data other than hosting the static files themselves. Being limited to only 8 requests per minute and 800 requests per day presents a bit of a challenge. In all likelihood, maybe 10 people ever visit the site, and that’s the extent of it. But there’s always the chance that a bunch of people as dumb as me share this, and suddenly I end up with more traffic than I expect, so having every visitor after the eighth within any 60 second period seeing a broken website is not ideal. Sigh Stonks.info needs a backend.
The Dumbest Possible Solution
How do we turn some arbitrarily large number of requests for some piece of information into a set number that we control? Why, that’s a cache, my friend. We can create some static resource to be the only thing our frontend knows about, and have a separate server-side task to update the contents of that resource on our own schedule. The dumb website can make as many requests as it likes to the resource to get the current value without causing any downstream API calls. All our stonksman needs to know is the current precent change in value of the Dow from the previous day’s close. So our static resource can be an incredibly simple bit of json:
{
"percent_change": -1.63329
}Stick this in an AWS S3 bucket (spoiler: that’s where the entire static frontend is going anyway) and we’re halfway* to a backend service!
*For some extremely small values of “half”
We don’t care in the slightest about the existing state of things, so this is a prime opportunity for a Lambda script. We can just assume that every time it’s triggered, it’s an atomic operation to ask Twelve Data what’s up with the Dow right now and write the relevant parts into our static json file in S3, leaving the frequency and timing of these updates to the scheduler. The bundled boto3 library makes this an almost trivial python script.
import boto3
import json
import urllib3
s3 = boto3.client('s3')
API_KEY = "lolimnottellingyougetyourownitsfree"
endpoint = 'https://api.twelvedata.com/quote?symbol=DJI&apikey=' + API_KEY
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
http = urllib3.PoolManager()
req = http.request('GET', endpoint)
json_data = json.loads(req.data)
extracted_data = {
"percent_change": float(json_data["percent_change"])
}
s3.put_object(Bucket='stonks.info',
Key='current.json',
Body=bytes(json.dumps(extracted_data).encode('UTF-8')))Twelve Data’s /quote API endpoint even does the math for us, including a percent_change key that’s exactly what we need.
Now we just need to come up with a schedule to trigger this thing. We know the NYSE isn’t open 24 hours or every day (cowards), so the value won’t change except during market hours, 9:30 AM - 4:00 PM EST (UTC-05:00).
AWS EventBridge (CloudWatch Events, before the wedding) lets us create an event rule given a cron schedule based on UTC, so following their particular weird cron syntax this should do what we need:
* 13-21 ? * MON-FRI *
We’ll run every minute between 13:00 and 21:00, to cover shifts between daylight savings time, on any day of the month, only on weekdays, any year. That totals up to 480 requests per day, well within the limit our generous and benevolent free API provides.
Unleash the Stonksman
I won’t get into every little detail of how I put together the specific HTML and CSS that draws this stuff; it’s pretty straightforward, and it’s all served statically on the site if you really want to look. It just took a little bit of editing the images from the original image macro meme, importing a very helpful CSS library called Animate.css, mixed with a splash of jQuery, and our friend is keeping us informed every minute about how The Stocks are doing.
Please, enjoy his hard work in real time here: https://stonks.info
Looking back at this now from 2021, maybe I was a bit naive with the ranges I chose, but at the time, a 1.5% single day change in value was a pretty uncommon event!
var threshholds = {
"NONE": [-Infinity, 0],
"LOW": [0.01, 0.7],
"MEDIUM": [0.71, 1],
"HIGH": [1.01, 1.5],
"INSANE": [1.51, Infinity]
};Ah, well after all, hindsight is 2020. Which is to say: a waking nightmare. Time to get back to my real investments.
