Marble Butterflies

week 1 | data

Such a lovely topic, nature, I expected to have loads of ideas once I started on February... But that didn't happen. After spending a bit too much time on January's visual, I had set myself a goal that I wanted to try and finish February within 20 hours, just to force myself to deliver something within a reasonable number of hours. And having a "dataviz" block wasn't helping me stay within that time limit 😞

However, at some point I thought about what defines nature for me, which was evolution and the randomness of mutations that in turn can lead to "better adaptations" to the environment. Which in turn reminded me of generative art and the semi-randomness which often guides those pieces of art. I've always loved generative art and thus suddenly February seemed like the perfect month to give it a try.

Now I only needed a subject to guide the randomness. The first thing I was drawn to were flowers, but damn, I could not find a single dataset about flowers that contained information such as the main color, average number of petals, blooming period, things like that. It was all images for image processing training or taxonomy lists (e.g. what group, family, etc. does the flower belong to). I therefore turned my attention to butterflies, for which I also have a fascination (don't ask my boyfriend how much time he's had to wait for me on vacations while I was trying to perfectly capture a butterfly, πŸ˜… ). But.... same problem, only taxonomy lists, no nice dataset with main wing color, size, etc.

The best (start of a) dataset that I could find, I kid you not, was eventually found on Gardens with Wing. It at least had some pages that divided butterflies into main colors, wing shape, size and species. So I manually turned the results of this pages filters into a dataset with 86 butterflies (found here).

The website of Gardens with Wings

The main color of a butterfly I could use to color whatever path the butterfly was going to have on my screen, the size of the butterfly could guide the thickness of the line/point, whatever I was going to make of it, and the species maybe could guide something else (the type of the path?). That wasn't a lot of information to use, but hopefully it would be enough 😁

week 2 | sketches

Once I figured out the butterfly and generative angle it seemed most appropriate to me to have the butterflies just flit around the screen. They always seem to fly quite randomly once they're near a plant going left, right, quite suddenly.

At first I thought about using the different path options to build up the butterfly's route, maybe Skippers would be straight lines and Swallowtails would be arcs. So that's what I drew below, but after having created a very rough version of the straight lines (1st image in the code section below) I knew that was not going to work :S However, after that point I didn't really go back to sketching, but continued to develop the paths by sketching with code, since it can be rather difficult to truly sketch the end result of random behavior, hehe.

Very rough sketch of the butterflies

I also drew some shapes because I wanted the front of each path to be visualized by a shape moving along, where each species would get its own shape. And the right page is really just questions or ideas I wanted to keep in mind while coding, such as "should a line (e.g. butterfly) quit after a certain time?"

week 3 & 4 | code

I knew there would we many many paths all going across the screen at once with probably no interactivity, so this month I finally did it right, I started with canvas instead of d3. For my very first attempt I just wanted the simplest thing in the screen, which in my mind were representing the butterfly "particles" by random straight paths, going in a different direction on each X milliseconds.

Very first result on screen with only straight lines

Well, that looked absolutely dreadful. Which instantly taught me that making random things look beautiful was difficult (and I got even more respect of people like Jared Tarbell and Anders Hoff).

I started from the sort-of generative example I had used in my "SVGs beyond mere shapes" presentation, which I adjusted from Sketch.js' particle demo to create my butterflies and how to use forces and drags to create random paths for them. On each animation frame one extra step would be added to the total path of the butterfly, thus giving the idea of them "flying over the screen".

The horrible screen-shot above showed me that I would probably need some smoothing of the curves, like d3's line interpolation functions. Thankfully, I could start from the code of this stackOverflow question that sort of draws a curved spline through the points that form the path of the butterfly. Below you can see some of my earliest tests, first just getting a smooth curve and once I figured that out I started to gently jitter the path of the butterflies after each frame to get a more random looking path. And once I also dropped the opacity a lot and got to the smoky bottom orange line in the image below I finally felt that this could turn out ok after all πŸ˜„

Tests for different butterfly paths

For ease I just let all my paths start from the center, but that did already create some interesting results once I let more butterflies free across the screen. I started to add some rules based on my actual butterfly data. Each frame a few butterflies are created, they are picked from the dataset of Β±50 butterflies (I filtered out the browns, that's just an ugly color). The color of the butterfly path is then defined by the main color of the butterfly. However, to get more diversity I used Tinycolor.js to pick randomly between 5 very similar colors (so for blue, it might go a bit more dark blue or a bit more aqua). The opacity and thickness of the path and the amount the path is "jittered" is governed by the size of the butterfly (I have small, medium and large). Naturally, large butterflies have thicker paths.

I then turned to the paths themselves. In my initial set-up, the paths were quite straight. A butterfly would dash over the screen never to be seen again. Well, to mimic a butterfly more I started to make more curved paths, but that took some effort to create in a semi-random fashion, that once a butterfly was curving towards a side it would do so for a bit of time before potentially starting to curve towards the other side. In my tests below I had actually also taken out the black butterflies, because I thought that color would also not look nice, but eventually I placed them back in. Also, the top left is when using a multiply blend mode on the canvas whereas for the others I had switched to screen

Investigating the curves in the butterfly paths

Only lines seemed a bit boring, so I also played around with different paths. The smallest butterflies now get a dotted line and for one particular species, the "Skippers" I implemented circles instead of a line. But again here I had to experiment, because jittering a set of circles around will eventually only result in a very thick blobby line. In the image below the circles were drawn thankfully, but somehow the underlying path was also being filled πŸ˜…

Doing something wrong when creating the circle paths

Alright, things were starting to look better and more diverse now. I played around with different bubble effects. I quite like the "scatter" approach of the screen-shot below, but this was taken just a few seconds after initialization. If you kept it running longer, the circles would completely disappear due to the lines overdrawing the circles every frame, whereas the circles were never drawn again. Thus in the end I decided to also keep drawing the circles over and over, but only give them a very small jitter and stroke them instead of filling them.

Taken a few seconds after reload so the bubble paths are still visible quite well

Instead of the butterflies starting from the center, I had them start from right outside the screen and fly in to get a better spread (however in the final version the script randomly chooses between the center/outside set-up because I couldn't choose 😁 ). For fun I added the hexagon + circle of datasketches' logo in the center and played around with getting a butterfly shape in the middle of that which still looked a bit random (but recognizable as a butterfly), which eventually took about 3-4 hours to figure out o_O

Having the paths start from outside the screen and adding a heagon & circle logo in the center

Using openType.js that Sarah, our guest for this month, recommended I could recreate a font as a canvas path and then also wiggle that around to get a less clean look which fit better with the overall style (although in some iterations the wiggling is so weird that you can't really read it anymore, haha)

The final thing that I tried was to also place markers at the start of each path, as if those were the butterflies (as I'd drawn in my sketches). The screen-shot below shows a test where I hadn't erased the earlier markers (here just circles), but it became clear quite fast that this was not going to be an improvement. The circles darting over the canvas were highly distracting and not really needed, in the first few seconds you can quite clearly see the different paths growing and twirling about, which was enough for me

Testing markers at the start of each butterfly path

After what seemed like endless tweaking of the paths to get better looking results I checked out my timing app and saw that I'd reached 19:45 hours for this month, so it was time to finish it off (with this write-up added I think I'll end somewhere around 21:30 hours). I added a link below that opens a version without all the white logo stuff on top and also a "button" (really only some text) in the bottom section to stop the animation, otherwise it just goes on an on eventually making the page white again.

I've called it "Marble Butterflies" because the smoky lines across the screen remind me of the patterns in marble. You can see the final result below, although due to the randomness, every version will be a completely unique piece of data art πŸ˜‰

The final version where the butterflies start from the center The final version where the butterflies start from just outside the screen

It was quite fun to not focus on creating something that should be "understood" in a sense, but just experiment with stuff and keep the things that look kind of cool. I'd like to continue to investigate the more artful or generative side of dataviz I guess πŸ˜ƒ

PS: There are also several high-quality giclΓ©e prints available of this project in my online shop!

san francisco

Wow I don't know how to begin. This month has taken me so long it's embarassing, but I'm also super glad it's done - huzzah! When we first decided on the topic for February, I had intended to do something with BBC's Planet Earth 2. I had even started getting data for the first episode, jotting down which animals appeared when. But it didn't really go any further than that. (I think I wasn't excited enough about the topic for the amount of manual data entry it required.)

More than a year later, I've finished a piece for SFMOMA that I'm happy can also double for datasketches. I have to admit it's still sort of cheating - the data isn't about Nature, it's about texts sent to SFMOMA. But I'm looking at individuals' texts and their interactions with SFMOMA, so I'm hoping it can fit under...human nature hehehe. And plus there are flowers! So nature! I'm very thankful that Nadieh was ok with my little tweak to our rules - thank you Nadieh!

week 1 | data

Last Last June, SFMOMA launched Send Me SFMOMA, a service where people could text a number requests like "send me love", or "send me art", and SFMOMA would respond with an artwork in their collection that best matched the request. They received 5 million texts from hundreds of thousands of people. Last October, they contacted me to do something with that data, and I was over the moon with excitement (bc SFMOMA!).

When I first started looking at the data, I did some simple exploration - what the attributes were, what keywords were most popular, what artworks came back the most often. I also knew that I wanted the end result to be a painting, bc I'm a cheesy individual and I wanted to make art out of art. The idea I ended up pitching to SFMOMA was to trace an individual's "journey" throughout the month, what positive or negative thing they might have asked for, what kind of artwork they got back. I wanted to see if I can experience what that person experienced, and feel what that person felt.

Once I had the idea, the data part was quite easy - they gave me a huge file of all the texts they ever received, and another huge 1TB file of all the texts they ever sent back. From that second file, I wrote a simple script to get the first 100 individuals. I ran the keyword of each text through a sentiment library, and my cousin Max (a junior in college studying CS) wrote a script to get additional metadata - including date, title, artists, country of origin, color palette, Shannon Entropy - for each artwork from the SFMOMA API.

As far as data collection, prep, and cleaning went, this was probably one of the most straightforward months I've had while doing datasketches, for which I'm really happy about.

week 2 | sketches

The very first thing I did once I had the data was to try and implement my own version of Tyler Hobb's watercolor effect. It didn't turn out quite well: (The actual tweet that has some great responses)

But thankfully, Taylor Baldwin came to the rescue with a link to his own implementation, and later on Tyler chimed in with a fix. I was able to get my version looking more like Tyler's (top left in the screenshots below), but once I plugged in the actual data (where each blob is a color from an artwork that an individual got back), it was a mess again and I knew I had to try a different approach more than just watercolor blobs:

Around the time I was first working on this project, I had moved to Tokyo for 3 months. Japan is a country that reveres nature and the change in seasons, and in March when I first arrived, flowers were everywhere. I remember walking past this ad in a subway station and stopping and double tracking back because I just loved the whimsy and simple beauty of the flowers: And I knew there and then that I wanted to work flowers and nature into the project (and when I also hatched the diabolical plan to turn a client project into a datasketches project yet again). As soon as I got back in front of my computer/laptop, I jotted down my ideas for how to incorporate flowers into the visualization. At the same time, I started looking at the SFMOMA API to see what kind of data I could get back about each artwork. A few days later, I worked on the flower petals, taking what I learned from July film flowers about SVG curve commands, as well as newly learned functions like P5's randomGaussian and noise to get a (hopefully) more natural flower petal shape: I especially loved the handdrawn effect I was able to get by using Perlin noise and varying the circle radius along the flower petal path, but unfortunately wasn't able to use it in the final visualization (it was just too computationally heavy especially when I was trying to draw dozens to hundreds of flowers).

week 3 & 4 | code

So this is one of those months that prove that even if the data part goes smoothly and I know exactly what I want to do, the actual execution (code) can take a really really long time.

This was my very first attempt, where each text was a flower placed around in a spiral, and was colored by a watercolor blob. Not only was it aesthetically unappealing, it was so computationally heavy my Macbook Pro (with 16GB RAM!) was choking up. I immediately moved the watercolor calculation into a Web Worker so they wouldn't clog up the UI. I also got rid of the handdrawn petal path effect, because it was too subtle to appreciate when the flowers were at such small sizes and wasn't worth the CPU that it'd take to calculate them.

I didn't know how to deal with the aesthetics, until I went to the Joint Graduation Exhibition of 5 Art Universities in Tokyo at the National Art Center and saw this piece: (I'm really sorry for the horrible quality of this photo, and for not remembering to take a picture of the artist's name and title of artwork.)

I was immediately taken by the cutout effect of the flowers, and I knew I wanted to try and recreate it: I liked this effect very much, where the watercolor effects could mix together and there were cutouts for each flower. Around this time I also realized that having a flower for every text was just too much, so I decided instead to only have a flower for each positive text that was sent. I tried circles and squares for the neutral and negative texts, and eventually ended up with leaves.

But I was still unhappy with the layout of the flowers - for individuals who sent a lot of texts, they would overlap or be pushed off the edge of the screen. I was stuck for weeks on how to best layout each text (and worked on other projects in the meanwhile), until cherry blossom season hit in Tokyo. Suddenly there was pink everywhere, and it was gorgeous and emotional (I had lived in Japan for 6 years as a child and cherry blossoms are amongst my fondest childhood memories) and I basically spent the whole week going to different spots admiring cherry blossoms.

It was on one of those walks through Shinjuku Gyoen (one of my favorite parks in Tokyo) that I started to realize, I've been trying to layout the flowers in random ways but actually, they're flowers so they grow on tree branches. And tree branches are just fractals, and that's a very straightforward, very much solved computer science problem. So I drew my flowers with branches, and I immediately loved it:

It took another two months after this point of tweaking to get to the final product. I added a slight curve to the branches to make them look a bit more natural, I made the flower petal shape less spiky, I fixed bugs...but most of all, I had a lot of trouble with the layout. I didn't like the amount of overlap of the flowers in the circular layout, so I moved to a horizontal layout: But with this layout a lot of the tree was cut off, so SFMOMA suggested a carousel sort of format: And I had it in this format for a while, but I really missed being able to see all the days of the week at the same time, and finally I adopted a layout with one day expanded and the rest smaller: And this layout made me the happiest, not only because I could see all the trees at the same, but also because it was implemented with CSS Grid, it was perfectly responsive.

I then put together a legend for how to read the visualization (also implemented in CSS Grid!) and an introduction to the project for the final version: I love this project so much because it had so many learnings and firsts: I learned Vue around this time, so all the data updates and interactions are managed in Vue. I successfully used a Web Worker, and I learned CSS Grid to layout all of the trees. It will also always remind me of all the experiences I had in Japan, since my stay there deeply influenced the form of this visualization. I'll always have little tweaks I want to do (I'm never 100% satisfied with my projects), but it's one of those projects where I'm really happy with the outcome.