Question:
What is 2015_1024 Code Challenge?
The 2015_1024 Code Challenge is a learning exercise for students attending Dan's Linux JavaScript Class at 3pm on 2015-10-24 at Hacker Dojo:
http://www.meetup.com/Dans-Linux-JavaScript-Class/events/225904559
We start with three tasks: Install Ubuntu, enhance it, and create account named ann:
Next, we learn some Emacs:
I see the buffer-menu as similar to the Mac-Dock, Ubuntu-Launcher or the Windows-Taskbar.
The buffer-menu is my favorite emacs feature; it allows my mind to QUICKLY switch focus among three types of objects:
- different shells
- different files
- different folders
The buffer-menu floats 'hot' shells, files and folders to the top.
This is useful behavior.
I mostly do 4 types of tasks on a laptop:
- Interact with file
- Interact with folder
- Interact with shell
- Interact with browser
I can do the first three types inside of emacs.
When I coordinate tasks with emacs rather than a Dock, I work much faster (because of the buffer-menu).
That sequence of creating a shell, and renaming it is the most difficult task I need to know.
All other emacs tasks can be done using mouse and arrow keys.
If I am an emacs power-user, I know two ways to use emacs to interact with folders.
The GUI-way is to click on the file-cabinet at the top.
Another way, which is quicker, is to type command ctrl-x then letter 'f'.
I should see something like this:
-
After I learn some Emacs or some other editor,
I Install Node.js in this folder: ~ann/node/
-
cd ~ann
wget https://nodejs.org/dist/v5.0.0/node-v5.0.0-linux-x64.tar.gz
tar zxf node-v5.0.0-linux-x64.tar.gz
rm -rf node
mv node-v5.0.0-linux-x64 node
-
Then, I add Node.js to PATH:
export PATH="/home/ann/node/bin:${PATH}"
echo 'export PATH="/home/ann/node/bin:${PATH}"' >> ~ann/.bashrc
-
Run a test:
which node
node -e 'console.log("hello world")'
-
Install CoffeScript to test npm:
which npm
npm install -g coffee-script
ls -la /home/ann/node/lib/node_modules/
which coffee
coffee -e 'console.log "hello coffee!"'
Now you are ready to do D3.js development on your laptop.
Try this shell command:
~ann/app22/http.bash
Browse this URL:
http://localhost:8080
The above URL should resemble the content served by this URL:
https://sjc408.herokuapp.com
If you want to deploy this repository to Heroku, follow this recipe:
- Create an account at heroku.com website
-
Download/Install Heroku client
cd ~ann
wget https://s3.amazonaws.com/assets.heroku.com/heroku-client/heroku-client.tgz
tar zxf heroku-client.tgz
echo 'export PATH=/home/ann/heroku-client/bin:${PATH}' >> ~ann/.bashrc
echo export PATH=/home/ann/heroku-client/bin:${PATH}
-
I created ssh-key for ann account (assuming ann has none yet):
ann@dev06:~$
ann@dev06:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ann/.ssh/id_rsa):
Created directory '/home/ann/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ann/.ssh/id_rsa.
Your public key has been saved in /home/ann/.ssh/id_rsa.pub.
The key fingerprint is:
7c:73:e0:26:8b:62:f6:a4:1d:78:22:e5:cd:3a:5c:eb ann@dev06
The key randomart image is:
+--[ RSA 2048]----+
| |
| |
| . |
| . . . |
| . S = . |
| o +.. = o |
| ..B.B.. |
| +oX.. |
| ooE |
+-----------------+
ann@dev06:~$
ann@dev06:~$
ann@dev06:~$
-
I used heroku-client to "login" my shell.
I gave a copy of ann public ssh-key to heroku:
heroku status
heroku auth:login
heroku auth:whoami
heroku keys:add
-
I used heroku-client to create a blank app named sjc408 at herokuapp.com
cd ~ann/app22
heroku create sjc408
-
git-push ~ann/app22 to heroku
cd ~ann/app22
git add .
git commit -am hellodojo
git push heroku master
-
I saw ~ann/app22 deployed at
https://sjc408.herokuapp.com
Challenge 10
Your challenge: understand what this HAML does:
!!!
%html(lang="en")
%head
%meta(content="text/html; charset=UTF-8" http-equiv="Content-Type")/
%title Mixing CRUD Idea with D3.js
%script(src="/d3.v3.min.js")
:css
body{font-style: sans-serif; font-size: 18px;}
%body
%h1 app22/crud.html
I use the CRUD acronym as a start when I want to learn about a new API.
%ul
%li The C in CRUD stands for Create.
%li The R stands for Retrieve.
%li The U stands for Update.
%li The D stands for Delete.
After I can use a given API to do CRUD operations on objects,
then I am on a path towards understanding that API.
%br/
%br/
When I do CRUD with D3.js I operate on two types of things.
%br/
%br/
First I CRUD HTML-elements like div-elements and svg-elements.
Also I CRUD JavaScript objects related to the D3.js API.
%h2 HTML CRUD:
%ul
%li Create Method 1: Type it in via emacs
%li Create Method 2: D3 call to append()
%li Retrieve: Just Click Reload Button
%li Update Methods: D3 calls: append(), attr(), style(), html(), text()
%li Delete Methods: D3 call: remove()
%h2 D3.js CRUD:
%ul
%li Create Method 1: obj1 = d3.select('ul#mylist');
%li Create Method 2: obj2 = obj1.selectAll('li');
%li Retrieve: Just list the object name (useful in debugger-console): obj2;
%li Update Methods 1: D3 calls: append(), attr(), style(), html(), text()
%li Update Method 2: D3 call: obj3 = obj2.data(myarray);
%li Create Method 3 (left over data into obj4): obj4 = obj3.enter();
%li Create Method 4 (left over HTML into obj5): obj5 = obj3.exit();
%li Delete Methods: D3.js offers no way to Delete a D3-object.
%hr/
#crudhere
%p CRUD happens.
%hr/
With debugger, I step through the JS in the script-element below to see CRUD in action:
:javascript
// I start by Create of D3 selection object:
var div_sel = d3.select('div#crudhere');
// Next I Create an HTML element:
div_sel.append('ul');
// More Creation:
var ul_sel = div_sel.select('ul');
li0_sel = ul_sel.append('li');
// Retrieve is simple but useful sometimes in debugger-console:
li0_sel;
// I Update an HTML element using D3 html() call:
li0_sel.html('<h1>I am Updated.</h1>')
h1_sel = li0_sel.select('h1');
// I can also Update using D3 text() call:
h1_sel.text('h1 has been updated by text().');
// The D in CRUD is Delete.
// I demo Delete of an HTML element after I create it:
li1_sel = ul_sel.append('li');
li1_sel.html('<h1>Delete Me.</h1>');
// This is a good place to stop the debugger so I see the h1-element before I delete it.
li1_sel.remove();
'done';
%br/
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/crud.html
Challenge 10 Questions:
- Which version of D3 is used?
- What does select() do?
- What does append() do?
- What does html() do?
- What does text() do?
- What does remove() do?
- Does remove() remove HTML, JS-object, or both, or neither?
- How to step through JS-code with browser-debugger?
- Which browser-debugger is better, Firefox or Chrome?
Challenge 11
Your challenge: understand what this HAML does:
!!!
%html(lang="en")
%head
%meta(content="text/html; charset=UTF-8" http-equiv="Content-Type")/
%title leftover_data.html
%script(src="/d3.v3.min.js")
:css
body {font-size: 18px; font-weight: bold; font-family: sans-serif;}
pre {background-color: #ddd;}
%body
%h1 app22/leftover_data.html
#refer
Refer to:
%br/
%a(href="http://bost.ocks.org/mike/circles/index.html" target="x")
http://bost.ocks.org/mike/circles/index.html
%hr/
In
%a(href="crud.html") crud.html
I describe the CRUD acronym to say that I can use D3.js to operate on HTML which resides in your browser.
%br/
%br/
The U in CRUD stands for Update.
%br/
%br/
A useful D3.js method which can Update HTML is the data() method.
%br/
%br/
The scenario below helps me understand the data() method.
%br/
%br/
I use emacs to create 4 li-elements:
%pre
%code <ul id='ul0'> <li>1</li><li>2</li><li>3</li><li>4</li> </ul>
Next, I create a JS array called myarray which has 4 values:
%pre
%code var myarray = [11,22,33,44];
Then, I create a D3 object named li_sel0 from the 4 li-elements:
%pre
%code var li_sel0 = d3.select("ul#ul0").selectAll("li");
Next, I 'join' myarray with li_sel0 using the data() method like this:
%pre
%code var joined_sel0 = li_sel0.data(myarray);
Then, I can copy values from myarray into the li-elements like this:
%pre
%code var updated_sel0 = joined_sel0.text(function(d){return d;});
So, this idea of using data in myarray to 'drive' an update of 4 li-elements is a central feature of D3.js and I think of it when ever I see calls to data() in D3.js syntax.
%br/
%br/
In the above scenario I ensured that both the ul-element and myarray had 4 members.
%br/
%br/
If, however, the ul-element has 4 members and myarray has 6 members then I enter into what I
call the leftover data scenario.
%br/
%br/
Assuming I have this HTML loaded into my browser:
%pre
%code <ul id='ul2'> <li>1</li><li>2</li><li>3</li><li>4</li> </ul>
Here is some JS to deal with that scenario:
%pre
%code
var urarray = [11,22,33,44,55,66];
var li_sel2 = d3.select("ul#ul2").selectAll("li");
var joined_sel2 = li_sel2.data(urarray);
I can see that I have two leftover members: [55,66]
%br/
%br/
What can I do with them?
%br/
%br/
The two likely scenarios are:
%ul
%li Ignore them
%li Create li-elements from them
Ignoring them is easy.
%br/
%br/
I create li-elements from them with a three-step process:
%pre
%code
var leftover_sel = joined_sel2.enter();
leftover_sel.append('li');
joined_sel2.text(function(d){return d;});
Notice the above call to the enter() method.
%br/
%br/
The enter() method, like data(), is a key feature of the D3.js API which allows data to drive the HTML.
%br/
%br/
When the data is static, then data() is useful. When the data
is dynamic and 'enters' an array, then the array becomes mismatched with the
corresponding HTML.
%br/
%br/
I can then use the enter() method help pull the new data into the HTML.
%br/
%br/
Feel free to right-click Inspect-Element in your browser to step your debugger through the code hidden below:
%hr/
%ul#ul0
%li 1
%li 2
%li 3
%li 4
%hr/
%ul#ul2
%li 1
%li 2
%li 3
%li 4
%hr/
:javascript
var myarray = [11,22,33,44];
var li_sel0 = d3.select("ul#ul0").selectAll("li");
var joined_sel0 = li_sel0.data(myarray);
var updated_sel0 = joined_sel0.text(function(d){return d;});
'Above, myarray and li_sel0 match cleanly';
var urarray = [11,22,33,44,55,66];
var li_sel2 = d3.select("ul#ul2").selectAll("li");
var joined_sel2 = li_sel2.data(urarray);
'Above, urarray has leftover data.';
var leftover_sel = joined_sel2.enter();
leftover_sel.append('li');
// fails: leftover_sel.text(function(d){return d;});
// works:
joined_sel2.text(function(d){return d;});
/ I create some whitespace below to help the debugger
%br/
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/leftover_data.html
Challenge 11 Questions:
Challenge 12
Your challenge: understand what this HAML does:
!!!
%html(lang="en")
%head
%meta(content="text/html; charset=UTF-8" http-equiv="Content-Type")/
%title leftover_html.html
%script(src="/d3.v3.min.js")
:css
body {font-size: 18px; font-weight: bold; font-family: sans-serif;}
pre {background-color: #ddd;}
%body
%h1 app22/leftover_html.html
#refer
Refer to:
%br/
%a(href="http://bost.ocks.org/mike/circles/index.html" target="x")
http://bost.ocks.org/mike/circles/index.html
%hr/
In
%a(href="crud.html") crud.html
I use the CRUD acronym as a perspective on the behavior of D3-objects inside your browser.
%br/
%br/
The U in CRUD stands for Update.
%br/
%br/
A useful D3.js method which can Update HTML is the exit() method.
%br/
%br/
The scenario below helps me understand the exit() method.
%br/
%br/
I use emacs to create 6 li-elements:
%pre
%code <ul id='ul0'> <li>1</li><li>2</li><li>3</li><li>4</li><li>5</li><li>6</li> </ul>
Next, I create a JS array called myarray which has 4 values:
%pre
%code var myarray = [11,22,33,44];
Then, I create a D3 object named li_sel0 from the 6 li-elements:
%pre
%code var li_sel0 = d3.select("ul#ul0").selectAll("li");
Next, I 'join' myarray with li_sel0 using the data() method like this:
%pre
%code var joined_sel0 = li_sel0.data(myarray);
Then, I can copy values from myarray into the li-elements like this:
%pre
%code var updated_sel0 = joined_sel0.text(function(d){return d;});
So, this idea of using data in myarray to 'drive' an update of 6 li-elements is a central feature of D3.js and I think of it when ever I see calls to data() in D3.js syntax.
%br/
%br/
The problem is I only have 4 values in myarray and the HTML I want to drive has 6 li-elements.
%br/
%br/
I have 'leftover HTML'.
%br/
%br/
What can I do with this 'leftover HTML'?
%br/
%br/
The two likely scenarios are:
%ul
%li Ignore the 'leftover HTML'
%li Remove the 'leftover HTML'
Ignoring it is easy.
%br/
%br/
I remove leftover li-elements with a two-step process:
%pre
%code
var leftover_sel = updated_sel0.exit();
leftover_sel.remove();
Notice the above call to the exit() method.
%br/
%br/
The exit() method, like data() and enter(), is a key feature of the D3.js API which allows data to drive the HTML.
%br/
%br/
When the data is static, then data() is useful. When the data
is dynamic and 'exits' an array, then the array becomes mismatched with the
corresponding HTML.
%br/
%br/
I can then use the exit() method help remove HTML which no longer matches data which has just 'exited'.
%br/
%br/
Feel free to right-click Inspect-Element in your browser to step your debugger through the code hidden below:
%hr/
%ul#ul0
%li 1
%li 2
%li 3
%li 4
%li 5
%li 6
%hr/
:javascript
var myarray = [11,22,33,44];
var li_sel0 = d3.select("ul#ul0").selectAll("li");
var joined_sel0 = li_sel0.data(myarray);
var updated_sel0 = joined_sel0.text(function(d){return d;});
'Above, I have leftover HTML (2 li-elements).';
var leftover_sel = updated_sel0.exit();
leftover_sel.remove();
// Now I should have 4 li-elements
/ I create some whitespace below to help the debugger
%br/
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/leftover_html.html
Challenge 12 Questions:
Challenge 13
Your challenge: understand what this HAML does:
!!!
%html
%head
%meta(charset="utf-8")/
:css
body {
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
}
.box {
font: 10px sans-serif;
}
.box line,
.box rect,
.box circle {
fill: #fff;
stroke: #000;
stroke-width: 1.5px;
}
.box .center {
stroke-dasharray: 3,3;
}
.box .outlier {
fill: none;
stroke: #ccc;
}
%body
%script(src="/d3.v3.min.js")
%script(src="box.js")
:javascript
var margin = {top: 10, right: 50, bottom: 20, left: 50},
width = 120 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
var min = Infinity,
max = -Infinity;
var chart = d3.box()
.whiskers(iqr(1.5))
.width(width)
.height(height);
d3.csv("morley.csv", function(error, csv) {
if (error) throw error;
var data = [];
csv.forEach(function(x) {
var e = Math.floor(x.Expt - 1),
r = Math.floor(x.Run - 1),
s = Math.floor(x.Speed),
d = data[e];
if (!d) d = data[e] = [s];
else d.push(s);
if (s > max) max = s;
if (s < min) min = s;
});
chart.domain([min, max]);
var svg = d3.select("body").selectAll("svg")
.data(data)
.enter().append("svg")
.attr("class", "box")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.bottom + margin.top)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")")
.call(chart);
setInterval(function() {
svg.datum(randomize).call(chart.duration(1000));
}, 2000);
});
function randomize(d) {
if (!d.randomizer) d.randomizer = randomizer(d);
return d.map(d.randomizer);
}
function randomizer(d) {
var k = d3.max(d) * .02;
return function(d) {
return Math.max(min, Math.min(max, d + k * (Math.random() - .5)));
};
}
// Returns a function to compute the interquartile range.
function iqr(k) {
return function(d, i) {
var q1 = d.quartiles[0],
q3 = d.quartiles[2],
iqr = (q3 - q1) * k,
i = -1,
j = d.length;
while (d[++i] < q1 - iqr);
while (d[--j] > q3 + iqr);
return [i, j];
};
}
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/boxplots/boxplots.html
Challenge 13 Questions:
- How would a Data Scientist describe a boxplot?
- Will a boxplot tell me if a distribution is asymmetric?
- Would a boxplot of a Gaussian distribution have circles?
-
This syntax:
csv.forEach(something);
expects something to be what?
-
This syntax:
if (error) throw error;
does what?
- Math.floor(x) does what?
- a1.push(x) does what?
- What is opposite of a1.push(x)?
- In JS, do I need to add 'then' when I call 'if'?
- In JS, do I need to add 'else' when I call 'if'?
- In JS, should a Boolean expression be inside parenthesis?
- In HTML what is an svg-element?
- In D3.js what does attr() do?
- When I call attr(), how many arguments should I supply?
- What is the g-element?
- In SVG when I transform an element, should I use an attribute, CSS, or JS?
- What does Math.random() do?
- What is interquartile range?
Challenge 14
Your challenge: understand what this HAML does:
!!!
%html
%head
%meta(charset="utf-8")/
:css
text {
font: 10px sans-serif;
}
%body
%script(src="/d3.v3.min.js")
:javascript
var diameter = 960,
format = d3.format(",d"),
color = d3.scale.category20c();
var bubble = d3.layout.pack()
.sort(null)
.size([diameter, diameter])
.padding(1.5);
var svg = d3.select("body").append("svg")
.attr("width", diameter)
.attr("height", diameter)
.attr("class", "bubble");
d3.json("flare.json", function(error, root) {
if (error) throw error;
var node = svg.selectAll(".node")
.data(bubble.nodes(classes(root))
.filter(function(d) { return !d.children; }))
.enter().append("g")
.attr("class", "node")
.attr("transform", function(d) { return "translate(" + d.x + "," + d.y + ")"; });
node.append("title")
.text(function(d) { return d.className + ": " + format(d.value); });
node.append("circle")
.attr("r", function(d) { return d.r; })
.style("fill", function(d) { return color(d.packageName); });
node.append("text")
.attr("dy", ".3em")
.style("text-anchor", "middle")
.text(function(d) { return d.className.substring(0, d.r / 3); });
});
// Returns a flattened hierarchy containing all leaf nodes under the root.
function classes(root) {
var classes = [];
function recurse(name, node) {
if (node.children) node.children.forEach(function(child) { recurse(node.name, child); });
else classes.push({packageName: name, className: node.name, value: node.size});
}
recurse(null, root);
return {children: classes};
}
d3.select(self.frameElement).style("height", diameter + "px");
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/bubbles/bubbles.html
Challenge 14 Questions:
- What is the structure of the data which feeds bubbles.haml ?
-
This syntax:
color = d3.scale.category20c();
does what?
-
This syntax:
var bubble = d3.layout.pack().sort(null).size([diameter, diameter])
does what?
-
This syntax:
d3.json("flare.json", function(error, root) {});
does what?
- In JavaScript, can a function call itself?
Challenge 15
Your challenge: understand what this HAML does:
!!!
%html
%head
%meta(charset="utf-8")/
:css
body {
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
margin: auto;
padding-top: 40px;
position: relative;
width: 960px;
}
button {
position: absolute;
right: 10px;
top: 10px;
}
.bullet { font: 10px sans-serif; }
.bullet .marker { stroke: #000; stroke-width: 2px; }
.bullet .tick line { stroke: #666; stroke-width: .5px; }
.bullet .range.s0 { fill: #eee; }
.bullet .range.s1 { fill: #ddd; }
.bullet .range.s2 { fill: #ccc; }
.bullet .measure.s0 { fill: lightsteelblue; }
.bullet .measure.s1 { fill: steelblue; }
.bullet .title { font-size: 14px; font-weight: bold; }
.bullet .subtitle { fill: #999; }
%button Update
%script(src="/d3.v3.min.js")
%script(src="bullet.js")
:javascript
var margin = {top: 5, right: 40, bottom: 20, left: 120},
width = 960 - margin.left - margin.right,
height = 50 - margin.top - margin.bottom;
var chart = d3.bullet()
.width(width)
.height(height);
d3.json("bullets.json", function(error, data) {
if (error) throw error;
var svg = d3.select("body").selectAll("svg")
.data(data)
.enter().append("svg")
.attr("class", "bullet")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")")
.call(chart);
var title = svg.append("g")
.style("text-anchor", "end")
.attr("transform", "translate(-6," + height / 2 + ")");
title.append("text")
.attr("class", "title")
.text(function(d) { return d.title; });
title.append("text")
.attr("class", "subtitle")
.attr("dy", "1em")
.text(function(d) { return d.subtitle; });
d3.selectAll("button").on("click", function() {
svg.datum(randomize).call(chart.duration(1000)); // TODO automatic transition
});
});
function randomize(d) {
if (!d.randomizer) d.randomizer = randomizer(d);
d.ranges = d.ranges.map(d.randomizer);
d.markers = d.markers.map(d.randomizer);
d.measures = d.measures.map(d.randomizer);
return d;
}
function randomizer(d) {
var k = d3.max(d.ranges) * .2;
return function(d) {
return Math.max(0, d + k * (Math.random() - .5));
};
}
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/bullet/bullet.html
Challenge 15 Questions:
- What is the structure of the data which feeds bullet.haml ?
-
This syntax:
var something = svg.append("g").append("text").text("hello");
does what?
- In above example, does "hello" end up in svg, or g, or text element?
-
This syntax:
g.attr("transform", "translate(-6," + height / 2 + ")");
does what?
-
This syntax:
g.call(chart);
does what?
-
This syntax:
d3.selectAll("button").on("click", function() {something;});
does what?
-
This syntax:
svg.datum(randomize)
does what?
-
This syntax:
Math.max(0, 4.4);
does what?
Challenge 16
Your challenge: understand what this HAML does:
!!!
%html
%head
%meta(charset="utf-8")/
:css
body {
font: 10px sans-serif;
shape-rendering: crispEdges;
}
.day {
fill: #fff;
stroke: #ccc;
}
.month {
fill: none;
stroke: #000;
stroke-width: 2px;
}
.RdYlGn .q0-11{fill:rgb(165,0,38)}
.RdYlGn .q1-11{fill:rgb(215,48,39)}
.RdYlGn .q2-11{fill:rgb(244,109,67)}
.RdYlGn .q3-11{fill:rgb(253,174,97)}
.RdYlGn .q4-11{fill:rgb(254,224,139)}
.RdYlGn .q5-11{fill:rgb(255,255,191)}
.RdYlGn .q6-11{fill:rgb(217,239,139)}
.RdYlGn .q7-11{fill:rgb(166,217,106)}
.RdYlGn .q8-11{fill:rgb(102,189,99)}
.RdYlGn .q9-11{fill:rgb(26,152,80)}
.RdYlGn .q10-11{fill:rgb(0,104,55)}
%body
%h1 Stock Market Visualization of Daily Deltas
%script(src="/d3.v3.min.js")
:javascript
var width = 960,
height = 136,
cellSize = 17; // cell size
var percent = d3.format(".1%"),
format = d3.time.format("%Y-%m-%d");
var color = d3.scale.quantize()
.domain([-.05, .05])
.range(d3.range(11).map(function(d) { return "q" + d + "-11"; }));
var svg = d3.select("body").selectAll("svg")
.data(d3.range(1990, 2017))
.enter().append("svg")
.attr("width", width)
.attr("height", height)
.attr("class", "RdYlGn")
.append("g")
.attr("transform", "translate(" + ((width - cellSize * 53) / 2) + "," + (height - cellSize * 7 - 1) + ")");
svg.append("text")
.attr("transform", "translate(-6," + cellSize * 3.5 + ")rotate(-90)")
.style("text-anchor", "middle")
.text(function(d) { return d; });
var rect = svg.selectAll(".day")
.data(function(d) { return d3.time.days(new Date(d, 0, 1), new Date(d + 1, 0, 1)); })
.enter().append("rect")
.attr("class", "day")
.attr("width", cellSize)
.attr("height", cellSize)
.attr("x", function(d) { return d3.time.weekOfYear(d) * cellSize; })
.attr("y", function(d) { return d.getDay() * cellSize; })
.datum(format);
rect.append("title")
.text(function(d) { return d; });
svg.selectAll(".month")
.data(function(d) { return d3.time.months(new Date(d, 0, 1), new Date(d + 1, 0, 1)); })
.enter().append("path")
.attr("class", "month")
.attr("d", monthPath);
//d3.csv("dji.csv", function(error, csv) {
d3.csv("GSPC.csv", function(error, csv) {
if (error) throw error;
var data = d3.nest()
.key(function(d) { return d.Date; })
.rollup(function(d) { return (d[0].Close - d[0].Open) / d[0].Open; })
.map(csv);
rect.filter(function(d) { return d in data; })
.attr("class", function(d) { return "day " + color(data[d]); })
.select("title")
.text(function(d) { return d + ": " + percent(data[d]); });
});
function monthPath(t0) {
var t1 = new Date(t0.getFullYear(), t0.getMonth() + 1, 0),
d0 = t0.getDay(), w0 = d3.time.weekOfYear(t0),
d1 = t1.getDay(), w1 = d3.time.weekOfYear(t1);
return "M" + (w0 + 1) * cellSize + "," + d0 * cellSize
+ "H" + w0 * cellSize + "V" + 7 * cellSize
+ "H" + w1 * cellSize + "V" + (d1 + 1) * cellSize
+ "H" + (w1 + 1) * cellSize + "V" + 0
+ "H" + (w0 + 1) * cellSize + "Z";
}
d3.select(self.frameElement).style("height", "2910px");
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/calview/calview.html
Challenge 16 Questions:
- What is the structure of the data which feeds calview.haml ?
- Is the next day observation to the right or in the square below?
- What is being shown in each square?
- Is it obvious that the Stock Market reverts to mean?
- Is it obvious that the Stock Market tends to trend?
-
This CSS syntax:
fill: #fff;
does what?
-
This CSS syntax:
stroke: #ccc;
does what?
-
This JS syntax:
d3.format(".1%")
does what?
-
This JS syntax:
var format = d3.time.format("%Y-%m-%d");
does what?
-
This JS syntax:
d3.scale.quantize()
does what?
-
This JS syntax:
d3.csv("GSPC.csv", function(error, csv) {something});
does what?
-
This JS syntax:
d3.nest().key(function(d) {something}).rollup(function(d) {something}).map(csv);
does what?
-
This JS syntax:
rect.filter(function(d) { return d in data; })
does what?
-
This JS syntax:
var t1 = new Date(t0.getFullYear(), t0.getMonth() + 1, 0)
does what?
-
The monthPath(t0) definition declares that monthPath() should be called with one parameter.
But in the above HAML, monthPath is called with no parameters.
Is that a bug or is monthPath being called correctly?
-
This JS syntax:
d3.select(self.frameElement).style("height", "2910px");
does what?
-
Usually when I call mysel.data(myarray) I understand that myarray should be an array.
When might I make a call like this:
mysel.data(function(d) {something})
?
Challenge 17
Your challenge: understand what this HAML does:
!!!
%html
%head
%meta(charset="utf-8")/
%title Crossfilter
:css
@import url(http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz:400,700);
body {
font-family: "Helvetica Neue";
margin: 40px auto;
width: 960px;
min-height: 2000px;
}
#body {
position: relative;
}
footer {
padding: 2em 0 1em 0;
font-size: 12px;
}
h1 {
font-size: 96px;
margin-top: .3em;
margin-bottom: 0;
}
h1 + h2 {
margin-top: 0;
}
h2 {
font-weight: 400;
font-size: 28px;
}
h1, h2 {
font-family: "Yanone Kaffeesatz";
text-rendering: optimizeLegibility;
}
#body > p {
line-height: 1.5em;
width: 640px;
text-rendering: optimizeLegibility;
}
#charts {
padding: 10px 0;
}
.chart {
display: inline-block;
height: 151px;
margin-bottom: 20px;
}
.reset {
padding-left: 1em;
font-size: smaller;
color: #ccc;
}
.background.bar {
fill: #ccc;
}
.foreground.bar {
fill: steelblue;
}
.axis path, .axis line {
fill: none;
stroke: #000;
shape-rendering: crispEdges;
}
.axis text {
font: 10px sans-serif;
}
.brush rect.extent {
fill: steelblue;
fill-opacity: .125;
}
.brush .resize path {
fill: #eee;
stroke: #666;
}
#hour-chart {
width: 260px;
}
#delay-chart {
width: 230px;
}
#distance-chart {
width: 420px;
}
#date-chart {
width: 920px;
}
#flight-list {
min-height: 1024px;
}
#flight-list .date,
#flight-list .day {
margin-bottom: .4em;
}
#flight-list .flight {
line-height: 1.5em;
background: #eee;
width: 640px;
margin-bottom: 1px;
}
#flight-list .time {
color: #999;
}
#flight-list .flight div {
display: inline-block;
width: 100px;
}
#flight-list div.distance,
#flight-list div.delay {
width: 160px;
padding-right: 10px;
text-align: right;
}
#flight-list .early {
color: green;
}
aside {
position: absolute;
left: 740px;
font-size: smaller;
width: 220px;
}
%body
#body
%a(href="https://github.com/square")
%img(height="31" src="logotype.png" width="122")/
%h1 Crossfilter
%h2 Fast Multidimensional Filtering for Coordinated Views
%p
%b Crossfilter
is a
%a(href="https://github.com/square/crossfilter") JavaScript library
for exploring large multivariate datasets in the browser. Crossfilter supports extremely fast (<30ms) interaction with coordinated views, even with datasets containing a million or more records; we built it to power analytics for
= succeed "," do
%a(href="https://squareup.com/register") Square Register
%p
Since most interactions only involve a single dimension, and then only small adjustments are made to the filter values, incremental filtering and reducing is significantly faster than starting from scratch. Crossfilter uses sorted indexes (and a few bit-twiddling hacks) to make this possible, dramatically increasing the perforÂmance of live histograms and top-
%i K
lists. For more details on how Crossfilter works, see the
= succeed "." do
%a(href="https://github.com/square/crossfilter/wiki/API-Reference") API reference
%h2 Example: Airline on-time performance
%p
The coordinated visualizations below (built with
= succeed ")" do
%a(href="http://mbostock.github.com/d3/") D3
%a(href="http://stat-computing.org/dataexpo/2009/") ASA Data Expo
dataset. The dataset is 5.3MB, so it might take a few seconds to download. Click and drag on any chart to filter by the associated dimension. The table beneath shows the eighty most recent flights that match the current filters; these are the
= succeed "," do
%i details on demand
%p
Some questions to consider: How does time-of-day correlate with
= succeed "?" do
%a(href="javascript:filter([null, [100, 150], null, null])") arrival delay
%a(href="javascript:filter([null, null, [1700, 2000], null])") longer
or
%a(href="javascript:filter([null, null, [0, 300], null])") shorter
flights more likely to arrive early? What happened on
= succeed "?" do
%a(href="javascript:filter([null, [80, 150], null, [new Date(2001, 0, 12), new Date(2001, 0, 13)]])") January 12
%a(href="javascript:filter([null, null, null, [new Date(2001, 0, 27), new Date(2001, 0, 29)]])") weekends
and
= succeed "," do
%a(href="javascript:filter([null, null, null, [new Date(2001, 0, 29), new Date(2001, 1, 3)]])") weekdays
%a(href="javascript:filter([[4, 7], null, null, null])") mornings
and
= succeed "?" do
%a(href="javascript:filter([[21, 24], null, null, null])") nights
%a(href="https://github.com/square/crossfilter/tree/gh-pages") Fork this example
and try your own data!
#charts
#hour-chart.chart
.title Time of Day
#delay-chart.chart
.title Arrival Delay (min.)
#distance-chart.chart
.title Distance (mi.)
#date-chart.chart
.title Date
%aside#totals
%span#active -
of
%span#total -
flights selected.
#lists
#flight-list.list
%footer
%span(style="float:right;")
Released under the
= succeed "." do
%a(href="http://www.apache.org/licenses/LICENSE-2.0.html") Apache License 2.0
Copyright 2012
%a(href="http://squareup.com") Square, Inc.
%a(href="https://github.com/square/crossfilter")
%img(alt="Fork me on GitHub" src="https://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" style="position: absolute; top: 0; right: 0; border: 0;")/
%script(src="crossfilter.v1.min.js")
%script(src="/d3.v3.min.js")
:javascript
// (It's CSV, but GitHub Pages only gzip's JSON at the moment.)
d3.csv("flights-3m.json", function(error, flights) {
// Various formatters.
var formatNumber = d3.format(",d"),
formatChange = d3.format("+,d"),
formatDate = d3.time.format("%B %d, %Y"),
formatTime = d3.time.format("%I:%M %p");
// A nest operator, for grouping the flight list.
var nestByDate = d3.nest()
.key(function(d) { return d3.time.day(d.date); });
// A little coercion, since the CSV is untyped.
flights.forEach(function(d, i) {
d.index = i;
d.date = parseDate(d.date);
d.delay = +d.delay;
d.distance = +d.distance;
});
// Create the crossfilter for the relevant dimensions and groups.
var flight = crossfilter(flights),
all = flight.groupAll(),
date = flight.dimension(function(d) { return d.date; }),
dates = date.group(d3.time.day),
hour = flight.dimension(function(d) { return d.date.getHours() + d.date.getMinutes() / 60; }),
hours = hour.group(Math.floor),
delay = flight.dimension(function(d) { return Math.max(-60, Math.min(149, d.delay)); }),
delays = delay.group(function(d) { return Math.floor(d / 10) * 10; }),
distance = flight.dimension(function(d) { return Math.min(1999, d.distance); }),
distances = distance.group(function(d) { return Math.floor(d / 50) * 50; });
var charts = [
barChart()
.dimension(hour)
.group(hours)
.x(d3.scale.linear()
.domain([0, 24])
.rangeRound([0, 10 * 24])),
barChart()
.dimension(delay)
.group(delays)
.x(d3.scale.linear()
.domain([-60, 150])
.rangeRound([0, 10 * 21])),
barChart()
.dimension(distance)
.group(distances)
.x(d3.scale.linear()
.domain([0, 2000])
.rangeRound([0, 10 * 40])),
barChart()
.dimension(date)
.group(dates)
.round(d3.time.day.round)
.x(d3.time.scale()
.domain([new Date(2001, 0, 1), new Date(2001, 3, 1)])
.rangeRound([0, 10 * 90]))
.filter([new Date(2001, 1, 1), new Date(2001, 2, 1)])
];
// Given our array of charts, which we assume are in the same order as the
// .chart elements in the DOM, bind the charts to the DOM and render them.
// We also listen to the chart's brush events to update the display.
var chart = d3.selectAll(".chart")
.data(charts)
.each(function(chart) { chart.on("brush", renderAll).on("brushend", renderAll); });
// Render the initial lists.
var list = d3.selectAll(".list")
.data([flightList]);
// Render the total.
d3.selectAll("#total")
.text(formatNumber(flight.size()));
renderAll();
// Renders the specified chart or list.
function render(method) {
d3.select(this).call(method);
}
// Whenever the brush moves, re-rendering everything.
function renderAll() {
chart.each(render);
list.each(render);
d3.select("#active").text(formatNumber(all.value()));
}
// Like d3.time.format, but faster.
function parseDate(d) {
return new Date(2001,
d.substring(0, 2) - 1,
d.substring(2, 4),
d.substring(4, 6),
d.substring(6, 8));
}
window.filter = function(filters) {
filters.forEach(function(d, i) { charts[i].filter(d); });
renderAll();
};
window.reset = function(i) {
charts[i].filter(null);
renderAll();
};
function flightList(div) {
var flightsByDate = nestByDate.entries(date.top(40));
div.each(function() {
var date = d3.select(this).selectAll(".date")
.data(flightsByDate, function(d) { return d.key; });
date.enter().append("div")
.attr("class", "date")
.append("div")
.attr("class", "day")
.text(function(d) { return formatDate(d.values[0].date); });
date.exit().remove();
var flight = date.order().selectAll(".flight")
.data(function(d) { return d.values; }, function(d) { return d.index; });
var flightEnter = flight.enter().append("div")
.attr("class", "flight");
flightEnter.append("div")
.attr("class", "time")
.text(function(d) { return formatTime(d.date); });
flightEnter.append("div")
.attr("class", "origin")
.text(function(d) { return d.origin; });
flightEnter.append("div")
.attr("class", "destination")
.text(function(d) { return d.destination; });
flightEnter.append("div")
.attr("class", "distance")
.text(function(d) { return formatNumber(d.distance) + " mi."; });
flightEnter.append("div")
.attr("class", "delay")
.classed("early", function(d) { return d.delay < 0; })
.text(function(d) { return formatChange(d.delay) + " min."; });
flight.exit().remove();
flight.order();
});
}
function barChart() {
if (!barChart.id) barChart.id = 0;
var margin = {top: 10, right: 10, bottom: 20, left: 10},
x,
y = d3.scale.linear().range([100, 0]),
id = barChart.id++,
axis = d3.svg.axis().orient("bottom"),
brush = d3.svg.brush(),
brushDirty,
dimension,
group,
round;
function chart(div) {
var width = x.range()[1],
height = y.range()[0];
y.domain([0, group.top(1)[0].value]);
div.each(function() {
var div = d3.select(this),
g = div.select("g");
// Create the skeletal chart.
if (g.empty()) {
div.select(".title").append("a")
.attr("href", "javascript:reset(" + id + ")")
.attr("class", "reset")
.text("reset")
.style("display", "none");
g = div.append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
g.append("clipPath")
.attr("id", "clip-" + id)
.append("rect")
.attr("width", width)
.attr("height", height);
g.selectAll(".bar")
.data(["background", "foreground"])
.enter().append("path")
.attr("class", function(d) { return d + " bar"; })
.datum(group.all());
g.selectAll(".foreground.bar")
.attr("clip-path", "url(#clip-" + id + ")");
g.append("g")
.attr("class", "axis")
.attr("transform", "translate(0," + height + ")")
.call(axis);
// Initialize the brush component with pretty resize handles.
var gBrush = g.append("g").attr("class", "brush").call(brush);
gBrush.selectAll("rect").attr("height", height);
gBrush.selectAll(".resize").append("path").attr("d", resizePath);
}
// Only redraw the brush if set externally.
if (brushDirty) {
brushDirty = false;
g.selectAll(".brush").call(brush);
div.select(".title a").style("display", brush.empty() ? "none" : null);
if (brush.empty()) {
g.selectAll("#clip-" + id + " rect")
.attr("x", 0)
.attr("width", width);
} else {
var extent = brush.extent();
g.selectAll("#clip-" + id + " rect")
.attr("x", x(extent[0]))
.attr("width", x(extent[1]) - x(extent[0]));
}
}
g.selectAll(".bar").attr("d", barPath);
});
function barPath(groups) {
var path = [],
i = -1,
n = groups.length,
d;
while (++i < n) {
d = groups[i];
path.push("M", x(d.key), ",", height, "V", y(d.value), "h9V", height);
}
return path.join("");
}
function resizePath(d) {
var e = +(d == "e"),
x = e ? 1 : -1,
y = height / 3;
return "M" + (.5 * x) + "," + y
+ "A6,6 0 0 " + e + " " + (6.5 * x) + "," + (y + 6)
+ "V" + (2 * y - 6)
+ "A6,6 0 0 " + e + " " + (.5 * x) + "," + (2 * y)
+ "Z"
+ "M" + (2.5 * x) + "," + (y + 8)
+ "V" + (2 * y - 8)
+ "M" + (4.5 * x) + "," + (y + 8)
+ "V" + (2 * y - 8);
}
}
brush.on("brushstart.chart", function() {
var div = d3.select(this.parentNode.parentNode.parentNode);
div.select(".title a").style("display", null);
});
brush.on("brush.chart", function() {
var g = d3.select(this.parentNode),
extent = brush.extent();
if (round) g.select(".brush")
.call(brush.extent(extent = extent.map(round)))
.selectAll(".resize")
.style("display", null);
g.select("#clip-" + id + " rect")
.attr("x", x(extent[0]))
.attr("width", x(extent[1]) - x(extent[0]));
dimension.filterRange(extent);
});
brush.on("brushend.chart", function() {
if (brush.empty()) {
var div = d3.select(this.parentNode.parentNode.parentNode);
div.select(".title a").style("display", "none");
div.select("#clip-" + id + " rect").attr("x", null).attr("width", "100%");
dimension.filterAll();
}
});
chart.margin = function(_) {
if (!arguments.length) return margin;
margin = _;
return chart;
};
chart.x = function(_) {
if (!arguments.length) return x;
x = _;
axis.scale(x);
brush.x(x);
return chart;
};
chart.y = function(_) {
if (!arguments.length) return y;
y = _;
return chart;
};
chart.dimension = function(_) {
if (!arguments.length) return dimension;
dimension = _;
return chart;
};
chart.filter = function(_) {
if (_) {
brush.extent(_);
dimension.filterRange(_);
} else {
brush.clear();
dimension.filterAll();
}
brushDirty = true;
return chart;
};
chart.group = function(_) {
if (!arguments.length) return group;
group = _;
return chart;
};
chart.round = function(_) {
if (!arguments.length) return round;
round = _;
return chart;
};
return d3.rebind(chart, brush, "on");
}
});
The above HAML is deployed at this URL:
https://sjc408.herokuapp.com/crossfilter/crossfilter.html
Challenge 17 Questions:
- What is the structure of the data which feeds crossfilter.haml ?
-
This CSS syntax:
@import url(http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz:400,700);
does what?
-
This CSS syntax:
#body {position: relative;}
does what?
-
This CSS syntax:
h1 + h2 {margin-top: 0;}
does what?
-
This CSS syntax:
line {shape-rendering: crispEdges;}
does what?
-
This CSS syntax:
rect.extent {fill-opacity: .125;}
does what?
-
What does this comment mean:
// (It is CSV, but GitHub Pages only gzip's JSON at the moment.)
?
-
This JS syntax:
d3.csv("flights-3m.json", function(error, flights) {something});
does what?
-
This JS syntax:
var nestByDate = d3.nest().key(function(d) { return d3.time.day(d.date); });
does what?
-
This JS syntax:
d.date = parseDate(d.date);
does what?
-
This JS syntax:
d.delay = +d.delay;
does what?
-
This JS syntax:
flights.forEach(function(d, i) {d.index = i;});
does what?
-
This JS syntax:
var flight = crossfilter(flights)
calls crossfilter().
Where is crossfilter(flights) defined?
-
What kind of structure does crossfilter expect flights to be in?
-
This JS syntax:
var all = flight.groupAll()
does what?
-
This JS syntax:
var date = flight.dimension(function(d) { return d.date; })
does what?
-
This JS syntax:
var dates = date.group(d3.time.day)
does what?
-
Does barChart() come from the D3.js API or is it part of crossfilter?
That could be considered an adequate code challenge for a two hour Meetup.
If you have questions, e-me:
bikle101@gmail.com
|