01 Feb 2023

Motivation and hypothesis

I’ve been recently appreciating the brilliant design and delivery ofThe Pudding. And after watching a video series of how they made just one of their posts, it got me thinking of a way to make this form of data journalism more accessible.

Ghost has been around for a bit, and I love the philosophy behind it: open-source and writer-centric. I want to bring the same sort of accessibility as Ghost, and Substack, and Medium to data journalism. Easy to pick up, powerful underneath, lets you monetize, track subscribers, readers, send emails - all of it.

As of right now, writers need to write their articles and develop their plots, diagrams, and visuals separately. Sure, they can mess around with D3 a bunch, but then you create super bespoke code just for that article.

What if there was a way to make it simpler and repeatable but without sacrificing how customizable it can be for the power user?

I’m calling it, temporarily, Journata (Journal + Data, I know it’s corny.)

How I plan to build it

I was tossing back and forth on whether to do this in SvelteKit or try Phoenix + Elixir again, but I settled on Svelte for a few reasons:

  1. It just hit 1.0 last month, so it is much more stable
  2. It brings a bunch of nice features to the table like transitions and tweened
  3. I don’t really have a need for real-time information, other than maybe a real-time dashboard
  4. I’ll most likely be doing a bunch of JS anyways

The absolute goal for this project is simplicity, like the last one. I want this project to be almost comically simple on the frontend, but with powerful functionality.

I’ve been looking into D3, and I think it’ll be helpful, but after reading throughthis blog post, it frankly looks like just doing it with Svelte components will work and allow me to leverage the power of Svelte.

That means, I will basically need to build a plotting library for SvelteKit.

(Why do I do this)

Soooo….here goes nothing.


February 1

I have fleshed out the basics of the site. I have a text editor that uses markdown thanks to thesvelte-markdown package.

A basic upload/drag a picture onto the page functionality is also in.

Verrrrryyy preliminary plotting is in. I’m talking points, scatter plot, and lines with some animations.

I should really draw out a diagram of how the data structures should look before I go too far - otherwise it’ll be spaghetti soon and linking all these components to scroll triggers will be difficult.

I do quite like the ability in svelte for all of the properties to be part of the component. That means that each of those will be accessible through a properties menu I will make where I can specify triggers for changes to that element and how to change it (e.g., change color, size, shape, location).

Overall, seems good so far!

One thing that is currently unknown is how to transport all this data to a backend - how do I serialize all of these objects and transitions, etc, in an intelligible way.

And how do I publish these sites? That is, how do I make it so the components are fixed in how they will appear, show, move, etc. without other people adjusting them? Maybe copying it and setting all the objects to be locked? Not sure.

February 2

Today I was basically trying to figure out how to get a scrollytelling element placed in. There was a svelte component someone made for scrollytelling, so I imported it.

However, I think I am going at this at the wrong direction.

I first need to get the visualizations working - get the property panel in place in order to adjust the visuals.

I looked around at visualization libraries, but what is sort of frustrating is that I still have to go in and adjust them to bake in the transitions and scrolly elements, or the plotting libraries are shipped so readily with their own CSS that it basically breaks the site.

So, what I am thinking is that I need to take inspiration from an existing library, add in scrolly and transition options to them, and see how it works.

That is, for each element on the page, there should be options for how to introduce them into the flow of scrolling. For instance, say there was a point - I should be able to say, without code: “this point should start to appear when I scroll here and it should appear using the “fly” transition, with a duration of 1 second.”

I suppose what I have in my head is similar to the timeline transition functionality that is present within powerpoint - the OG of visualization.

Tomorrow will be starting from this point and seeing where it takes me.

February 5

So, yesterday, I foundLayer Cake. It’s a svelte graphics framework, and I think it is exactly what I need. It doesn’t just spit out plots like the others, but allows me the flexibility of what I want to do.

I managed to get a rough working example of a scrolly element with transitions for a plot - the kind of thing you’d see on some blogging platforms, maybe.

Scrolly demo

Although the gif is rather unimpressive at first glance, it’s actually a great proof of concept. I was able to:

In essence, I was able to control just about everything about this plot almost for free. The next step will be including a property pane that exposes all of this functionality to me, letting me mess around with it without coding.

I can see it coming together already!

February 7

I think I need to really start to embrace the open-source, someone already did that thing, essence. Today, I discoveredQuill, which seems to be the answer to my prayers for a wide variety of things, mostly the editor.

What I think will need to happen, however, is I will need to extend the functionality a bit to include data visualization objects and their respective data. Not sure how to do that yet, but I will look into it tomorrow.

February 8

Decided to take a diversion today and begin to create a professional blog with example data visualizations that I intend to use Journata, or whatever the hell I decide to name it. Hopefully, I will run into plenty of bumps in creating the visualizations to give me good insight for how to better build it out.

So, far, a combination of a CMS and an editor like that of Ghost, would be delightful, like the WYSIWYG editor I found yesterday. Extending that functionality to include scrolly functionality, and plots, and any arbitrary input would be great.