D3 - Part 1 Intro
This series explores the popular JavaScript data visualization library d3 through the lens of ClojureScript. The d3 philosophy nicely fits the ClojureScript-Reagent-React functional paradigm. If you wish to narrate interactive stories with visualized data, d3 might be right for you. Let's dive right in...
Assume you want to visualize the following bar-data
{:bar-data
[{:key 1, :value 73}
{:key 2, :value 48}
{:key 3, :value 66}
{:key 4, :value 68}
{:key 5, :value 62}]}
The data above is essentially a hash-map having 1 key :bar-data with it's value as a vector of 5 hash-maps. We will visualize each of the 5 members in 5 different div with each div's width proportional to its :value.
Example-1: Barchart made purely from html divs.
The main logic is coded in the function d3-horizontal-bars. Don't be thrown off if you are seeing ClojureScript syntax for the first time. ClojureScript uses Lisp's s-expression syntax. This syntax has huge payoffs when you use it to build UIs. You can get a quick tour of the syntax here.
What is (+ 2 (- 7 4)) ? If you can run the computation in your head and arrive at the answer 5, you already know s-expressions.
(defn d3-horizontal-bars [example-data root-selection]
(let [outer-div (-> (d3/select root-selection))
{:keys [bar-data]} @example-data
display-data bar-data
width (-
(-> js/document
(.getElementById "example-1")
(.-clientWidth))
50)
x-scale (-> d3
.scaleLinear
(.domain
#js [0 (apply max (map :value display-data))])
(.range #js [0 width]))
color-scale (-> d3
(.scaleOrdinal (-> d3 (.-schemeTableau10)))
(.domain (clj->js (map :key display-data))))]
(.log js/console "in logic")
(-> outer-div
(.selectAll "div")
(.data (clj->js display-data))
(.join "div")
(.style "background" (fn [d] (color-scale (.-key d))))
(.style "border" "1px solid white")
(.style "font-size" "small")
(.style "color" "white")
(.style "text-align" "right")
(.style "padding" "3px")
(.style "width" (fn [d] (str (x-scale (.-value d)) "px")))
(.text (fn [d] (.-value d))))))
Notes Example-1
This example is hosted in a div example-1. d3-example-1 is its only child. d3-example-1 in turn has 5 child divs, each representing the 5 entities of bar-data.
bar-data, display-data, width, x-scale and color-scale are all local variables.
bar-data de-references the global reagent atom, passed as an argument. The syntax you see here is called destructuring. It is a convenient way to bind one or more lexical local variables (think lvalue of an assignment statement in any programming language as an approximation) from complex nested data (think rvalue of an assignment).
display-data is an alias to bar-data.
width uses ClojureScript <-> JavaScript interop to avoid hard coding and instead get the width of the parent div hosting the final script
Scales are convenience functions.
Say your chart members span values from 21 K to 37 K. You want to visualize the same over a range of pixels that run down from 1500 px to 800 px. Pixels run down if say for example, a bar graph is oriented vertically. Higher the y coordinate value, lower it's on the screen. Hence for vertical bar graphs, higher the y coordinate, lower the domain value. Scales can let you specify things like the above declaratively. In the made up example, our domain is [21000, 37000] and range is [1500, 800]. The scale itself can be specified as linear.
(let [scale
(-> d3 .scaleLinear
(.domain #js [21000 37000])
(.range #js [1500 800]))]
(scale 25000))
=> 1325
So the value 25K, corresponds to pixel 1325. Which makes sense as its nearer to 1500 than it is to 800.
Apart from linear scales, other scales are also available like ordinal scale and band scale. For these scales, the domain is the array enumeration of all your member's unique identifiers, like key of bar-data. Its specified as [1 2 3 4 5] and the value could be something like the d3 constant d3.tableau10. I always have trouble choosing colours. If you are like me, you may use an ordinal scale with tableau10 to pick unique colours for upto 10 members without obsessing too much about it.
The following picks a colour for the 8th member
(let [scale
(-> d3 (.scaleOrdinal (-> d3 .-schemeTableau10))
(.domain #js [0 1 2 3 4 5 6 7 8 9]))]
(scale 7))
=> "#ff9da7"
see how the colours repeat after all 10 colours are exhausted
(let [scale
(-> d3 (.scaleOrdinal (-> d3 .-schemeTableau10))
(.domain #js [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]))]
(map #(scale %) (range 15)))
=> ("#4e79a7" "#f28e2c" "#e15759" "#76b7b2" "#59a14f" "#edc949" "#af7aa1" "#ff9da7" "#9c755f" "#bab0ab" "#4e79a7" "#f28e2c" "#e15759" "#76b7b2" "#59a14f")
d3/select and selectAll are select operators. Through these operators it's easy to specify how you want the dom elements selected and transformed.
The d3 API avoids loops. This is where it gets weird. Once you select a bunch of nodes through selectAll, you can specify what you want done as if you are inside a pseudo body of a foreach loop.
In our example we specify many things we want done to the dom through the selection. These include binding our data set over all children divs, invoke the join operator with hardcoded string value "div" and a bunch of mutations we want done to each of the 5 children divs. These are things like setting the background, border etc.
This is typically done thru' code like (.style "font-size" "small") or (.style "background" (fn [d] (color-scale (.-key d))))
You may specify a hard coded value like "small" if you want the attribute to be the same for all 5 divs. If you want each member to be different, you specify a function that gets the data member as input. The return value of the function will then be used to dynamically set the attribute. Inside the function, you can inspect the member and can assign a member specific value, like the background in the above case. The (.-key d) is again js interop to read property value of a json object.
About selections and joins...
In this example, we are invoking the join operator through a hard coded value ("div"). But that's not all you can do. D3's data join lets you specify exactly what happens to the DOM as the data changes. This makes it possible to build amazing narratives for eg. animate data as it enters, updates and exits the dom emphatically. But for this, you should not use the hard coded value in the join operator. We will explore these later.
Another thing to keep in mind is that we bind data only after applying an interop like (clj->js display-data). This is because the d3 data operator expects values in JavaScript json objects and arrays which are different from ClojureScripts maps and vectors.
Boilerplate ClojureScript reagent imports and data to be visualized
We are integrating the d3 version 6.7.0 for this series. Each example uses its own global reagent atom like example-1-data.
(ns reframetemp.frontend.app
(:require [reagent.core :as r]
[reagent.dom :as rd]
["d3" :as d3]))
Boilerplate reagent set up
(defn d3-simple-example [example-data root-selection]
(let [div-id (keyword (str "div" root-selection))]
(r/create-class
{:reagent-render
(fn [] (let [v (:bar-data @example-data)] [div-id]))
:component-did-mount (fn [] (d3-horizontal-bars example-data root-selection))
:component-did-update (fn [] (d3-horizontal-bars example-data root-selection))})))
(defonce example-1-data
(r/atom
{:bar-data
[{:key 1, :value 73}
{:key 2, :value 48}
{:key 3, :value 66}
{:key 4, :value 68}
{:key 5, :value 62}]}))
(defn example-1 []
[:div
[d3-simple-example example-1-data "#d3-example-1"]])
(defn example-2 []
[:div
... and so on
(defn start []
(rd/render [example-1] (.getElementById js/document "example-1"))
(rd/render [example-2] (.getElementById js/document "example-2"))
... and so on
)
(defn ^:export init []
(start))
(defn ^:export init []
(start))
Boilerplate skeleton html
<!doctype html>
<html lang="en">
<head>
<meta charset='utf-8'>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>hello reagent</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.1/css/bulma.min.css"> <link rel="icon" href="https://clojurescript.org/images/cljs-logo-icon-32.png">
</head>
<body>
<div class="columns">
<div id="example-1" class="column is-full" >
</div>
</div>
<div class="columns">
<div id="example-2" class="column is-full" >
</div>
</div>
<div class="columns">
<div id="example-3" class="column is-full" >
</div>
</div>
<div class="columns">
<div id="example-4" class="column is-full" >
</div>
</div>
<script src="/js/main.js"></script>
<script>reframetemp.frontend.app.init();</script>
</body>
</html>
Example-2: Next we make the chart a bit interactive
The hello world example is fine. But not enough. The fundamental reason is we are simply not exploiting the browser's ability to run code. With a library like d3, we can make our data come alive.
With the boiler plate out of the way, in example 2 we are now ready to add some user affordances to the chart. We do so by adding a button and making its click event, mutate example-2-data
(defonce example-2-data
(r/atom
{:bar-data
[{:key 1, :value 73}
{:key 2, :value 48}
{:key 3, :value 66}
{:key 4, :value 68}
{:key 5, :value 62}]}))
(defn example-2 []
[:div
[d3-simple-example example-2-data "#d3-example-2"]
[:br]
[:div.buttons
[:button.button.is-dark
{:on-click
#(swap! example-2-data
(fn [d]
(update-in d [:bar-data]
(fn [s]
(map
(fn [d] (assoc-in d [:value]
(inc (rand-int 100)))) s)))))} "rand-int"]]])
Notes Example-2
The key difference is we add button functionality by coding the (defn example-2 [] hiccup.
Here is where ClojureScript shines over its rivals. Any function returning a hiccup is a react component. A hiccup is nothing but UI DOM represented as nested arrays and maps. Since a function can call other functions, a react component may enclose other react components. That's it. Nothing more to learn like template syntax and JSX syntax. So we have composiobility of UI components thru' functions and the ability to refactor dom, through hiccup data manipulation. All the tools and techniques you have learnt to manipulate arrays, sequences and maps are still relevant as you code the UX of an application.
Back to our example, we add the button code thru' hiccup syntax. The button mutates the data our example-2 is managing and changes the value of each member by assigning it a random number between 1 and 100.
inc works like this. Both the below are identical
((fn [x] (+ 1 x)) 29)
;=> 30
(inc 29)
;=> 30
=> 30 for both expressions
update-in works like this. You need to specify a function with arity 1 in order to use update. Use update, if you want your new value in a map, to be some function of its existing value
(let [d {:age 29}] (update-in d [:age] inc))
=> {:age 30}
assoc-in works like this. You need to specify a value while using assoc. Use this to associate a value into a key of a map, if you don't care about what was already there before the operation.
```
:::clojure
(let [test-vector [{:key 1, :value 74} {:key 2, :value 48} {:key 3, :value 66}
{:key 4, :value 68} {:key 5, :value 63}]]
(map (fn [d] (assoc-in d [:value] (rand-int 5))) test-vector))
=> ({:key 1, :value 4} {:key 2, :value 3} {:key 3, :value 0} {:key 4, :value 3} {:key 5, :value 0})
What you are seeing in example-2, is adding the button ui styling and what should happen on its click event by combining update-in and assoc-in as, we want the mutation to happen 1 level deep. The logic is straightforward and works like the above examples illustrate.
Example-3: Next we make the bars move
(defn d3-horizontal-bars-with-transition [example-3-data root-selection]
(let [outer-div (-> (d3/select root-selection))
{:keys [bar-data]} @example-3-data
display-data bar-data
width (-
(-> js/document
(.getElementById "example-1")
(.-clientWidth))
50)
x-scale (-> d3
.scaleLinear
(.domain
#js [0 (apply max (map :value display-data))])
(.range #js [0 width]))
color-scale (-> d3
(.scaleOrdinal (-> d3 (.-schemeTableau10)))
(.domain (clj->js (map :key display-data))))]
(.log js/console "in logic")
(-> outer-div
(.selectAll "div")
(.data (clj->js display-data))
(.join "div")
(.transition)
(.duration 2000)
(.style "background" (fn [d] (color-scale (.-key d))))
(.style "border" "1px solid white")
(.style "font-size" "small")
(.style "color" "white")
(.style "text-align" "right")
(.style "padding" "3px")
(.style "width" (fn [d] (str (x-scale (.-value d)) "px")))
(.text (fn [d] (.-value d))))))
Notes Example-3
Just these 2 lines, bring about a dramatic effect. The payoff has begun. Appreciate how smoothly each member div is interpolated from existing width to new width !
(.transition)
(.duration 2000)
Example-4: We add 2 more buttons to inc and dec the values
(defonce example-4-data
(r/atom
{:bar-data
[{:key 1, :value 73}
{:key 2, :value 48}
{:key 3, :value 66}
{:key 4, :value 68}
{:key 5, :value 62}]}))
(defn example-4 []
[:div
[d3-example example-4-data "#d3-example-4"]
[:br]
[:div.buttons
[:button.button.is-dark
{:on-click
#(swap! example-4-data
(fn [d]
(update-in d [:bar-data]
(fn [s]
(map
(fn [d] (assoc-in d [:value]
(inc (rand-int 100)))) s)))))} "rand-int"]
[:button.button.is-dark
{:on-click
#(swap! example-4-data
(fn [d]
(update-in d [:bar-data]
(fn [s]
(map
(fn [d] (update-in d [:value]
inc)) s)))))} "inc"]
[:button.button.is-dark
{:on-click
#(swap! example-4-data
(fn [d]
(update-in d [:bar-data]
(fn [s]
(map
(fn [d] (update-in d [:value]
dec)) s)))))} "dec"]]])
Notes Example-4
Just shows how easy it is to add more agency. We add 2 more buttons that mutate the example's data by incrementing the value and decrementing the value by 1. That's a lot of bang for minimal code. It's declarative, expressive and it almost feels like nothing can go wrong if we progressively deal with such functional units. This is the promise of functional programming.
This concludes our Part 1 of this series.