Extra challenges

This page contains several advanced exercises that cover D3's brushing, dragging, and zooming functionality. These exercises will not be graded and no points will be awarded for them. But, if you are up for it, they are a fun challenge!

Network visualisations

We have prepared a force-directed network visualization that shows the interactions of characters in Star Wars. As in previous exercises, the data is loaded in ex_1.svelte which then constructs a _ex_1_network.svelte component that contains the actual visualization. Aside from the forceSimulation and the way images are used as fill for some of the circles, most things should look familiar at this point. The forceSimulation, however, deservese an explanation: 

const simulation = forceSimulation(nodes)
    .force('link', forceLink(links).id((d) => d.label))
    .force('charge', forceManyBody())
    .force('collide', forceCollide((d) => rScale(d.value)))
    .force('center', forceCenter(width / 2, height / 2))
    .on('tick', () => {
      // Needed so that svelte detects the values have changed...
      nodes = nodes;
      links = links;
    });

We use d3.js' d3-force module for running force simulations, which compute how a network should be layed out so that it is most legible. Essentially, the simulation tries to minimize the forces we introduce. Several forces are provided in the module, we used the:

To minimize the forces in the layout, the simulation uses a temperature schedule that is controlled throught the .alpha() helper function. Initially the temperature is high, allowing the nodes to move faster. As the simulation runs, the temperature decreases, until it hits zero. Then the simulation stops.

While the simulation is running, it will continually release a tick event. We can monitor that event to re-draw the nodes and edges while the simulation is running. In Svelte, our tick callback actually only has to tell Svelte that the values within the nodes and links arrays have changed. We do that by assigning both arrays to themselves, as Svelte uses assignment to detect changing variables. The simulation will have added and update several fields on the nodes and links:

In the following exercises you will extend the visualization with three interactions:

  1. Support dragging nodes so a user can manipulate the layout.
  2. Support zooming and panning to better inspect the network.
  3. Highlight the nodes and links connected to a node when you hover over a node.

You will develop these interactions in separate JavaScript files ( _ex_1_drag.js, _ex_2_zoom.js, and _ex_3_hover.js). This allows us to re-use some of the functionality later on. However, we will have to do a bit more work to use Svelte's reactivity. But that is worth the trouble!

Exercise 1 (0 pt.)

For this exercise, you have to edit the files src/routes/extra_challenges/exercises/_ex_1_network.svelte and src/routes/extra_challenges/exercises/_ex_1_drag.js. Any changes you make to those file should show up below.

d3.js provides the d3-drag module to add dragging functionality to DOM-elements. As you probably expect by now, to use the module, you have to import, call, and configure the drag() function. This gives back a function that takes a selected DOM-handle and makes that element draggable. Like with the brush in exercise 3 of week 7, you can configure what has to happen when the drag starts, moves, and ends.

For a nice dragging behaviour on force-directed networks, you have to increase the temperature of the simulation to 0.3 when a drag starts and keep it at that level while the drag moves. This ensures the layout gets a chance to adapt. To actually move a node, you have to force it to the position of the mouse while the drag moves. Hint: check the argument your callbacks are given when called! Finally, when the drag ends, you have to release the node. The simulation will continue for a short while settling in to the new layout, you do not have to stop it when the drag ends.

With this information, implement a re-usable dragging function in _ex_1_drag.js. The file should export a function that takes a force-simulation as argument and gives you a function to use as an action on the circle elements. That action-function should take each circle's node variable as second argument. Configure the dragging behaviour within the action-function as described and use it in _ex_1_network.svelte. Remember that an action-function is a function that takes a handle to an element on the DOM as argument. For example, when you specify <circle use:func /> Svelte will call func(circleHandle) when the circle is constructed. To pass more arguments to an action-function, you can use <circle use:func={value} />, which will call func(circleHandle, value).

Loading the data, please wait...

Exercise 2 (0 pt.)

For this exercise, you have to edit the files src/routes/extra_challenges/exercises/_ex_2_network.svelte and src/routes/extra_challenges/exercises/_ex_2_zoom.js />. Any changes you make to those file should show up below.

For zooming and panning, we will use d3.js' d3-zoom module. Using this module is similar to the brush and drag modules. To configure it, you have to specify the size of the SVG element using the .extent() function and register a callback for the zoom event. This event is released when the user zooms or drags on the element you applied the zoom-function to. The transform property of the argument that is passed to your callback specifies the SVG transform that corresponds to the user's movement. We want to capture that transform and have Svelte automatically use new values when they occur.

If we were to implement this functionality directly within _ex_2_network.svelte, you could use the same approach as with the brush in Exercise 3 (where we updated the range variable and had svelte react to the changes). For this exercise, however, you have to implement it in _ex_2_zoom.js. Svelte cannot monitor if variables in other files change, except when you use a store. For this exercise, you will use a readable store: 

const myStore = readable(new Date(), function start(set) {
  const handle = setInterval(set(new Date()), 1000);
  return function stop() {
    clearInterval(handle);
  }
});

Readable stores are used to give acces to values that change, but cannot be changed by the components that use them. For example, they can store the current time, as in the example above. Here, we created readable store with current time as initial value. The second argument is a function that is called the moment a first component starts using the value of the store. In that function, we update the value of the store every second by using the set function that is given as argument to the start function. Finally, the start function returns a function that performs any cleanup that has to happen when the last component stops using the store. Here, we cleared the interval so that the store's value is not updated every second anymore.

Using this information, implement zooming and panning functionality in _ex_2_zoom.js. The file should export a function that takes the width and height of the element it will be applied to and return an action-function that configures the zoom-module. Define a readable store that monitors the transform value given in the zoom event callback. Hint: you need to use the store's set function in the zoom callback! Assign the store as a property of the action-function, so that it can be accessed later on. Hint: Functions in JavaScript are really just objects that you can call, so you can give properties using: 

function func() {
  return 'func called!';
}
func.myProperty = 'func property accessed!';
console.log(func());
console.log(func.myProperty);

Use the zoom-action you have created on the SVG-element in _ex_2_network.svelte. Apply the transform to the SVG group tag that contains the nodes and links and not on the SVG element. This makes sure that the Star Wars logo does not move when you zoom or pan!

Loading the data, please wait...

Exercise 3 (0 pt.)

For this exercise, you have to edit the files src/routes/extra_challenges/exercises/_ex_3_network.svelte and src/routes/extra_challenges/exercises/_ex_3_hover.js />. Any changes you make to those file should show up below.

The hover interaction is a bit different from the other two, as we do not need to use any d3.js functionality. Instead we can just use Svelte's on:mouseover and on:mouseout directives to capture the user's interaction. The complexity of this interaction is in determining which nodes and links should be highlighted and which ones should fade.

Because you have to implement the hover functionality in _ex_3_hover.js, you will have to use stores again to make Svelte detect when values change. In this case, you will need both writable and derived stores. writable stores are the easiest type of store in Svelte, you can just initialize them with a value and then assign to them using the $ notation: 

// Create the writable store
const myStore = writable('initial value');
// Update the value
$myStore = 'new value!';
// Print the value when the value changes
$: console.log($myStore);

derived stores can be used to compute values that depend on the value of other stores. For instance, if we have a store containing the grades of a student, then a derived store can be used to automatically update that student's average grade: 

import { mean } from "d3-array"; 
const average_grade = derived(grades, ($grades) => {
  return mean($grades);
});

Here, a derived store is constructed by giving it the store that it depends on and a callback function that uses that store's value to computed the derived store's value. Specifically, grades is a store containing the grades of a student as an array of numerical values and the callback computes the mean grade for that student. It is a convention to name the variable within the callback as the name of the store with a $ prefixed (i.e., $grades), but the $ does not have any special meaning here. It just indicates that you are working with the value of a store and not the store object itself!

Aside from .filter(), which we showed before, two additional array manipulation functions are often useful in derived stores: .map() and .reduce(). They are used to replace typical for-loop patterns. The .map() replaces a loop that creates a single value for each item in an array. For example, 

const xs = nodes.map(d => d.x);

creates an array containing the x-coordinates of the nodes.

.reduce() is more flexible, as it does not have to return a value for every item in the array. Instead, you get control over what to add to the output and what the output should be. For example, a .reduce() can be used to sum all elements in an array: 

const sum = array.reduce((output, current_value) => {
    return output += current_value;
  }, 0);

Here, the first argument is a callback that is called on every item in the array. This callback takes as arguments the variable that will be the output and the current value in the array. Within this callback you can update the output variable, here we simply add the current value to it. The callback has to return the updated output variable, as it will be used in the next iteration. The second argument is the output's initial value. In this example we used zero as initial value. A .reduce() can also be used to convert an array into an object, which is sometimes useful: 

const object = array.reduce((output, current_item) => {
    output[current_item.id] = current_item;
    return output;
  }, {});

Note, objects cannot be used as keys in normal JavaScript objects, that is why we used the id field in this example!

Using this information, complete the hover functionality in _ex_3_hover.js. The file already exports a function that takes the nodes and links arrays, computes each node's neighbors, and contains a writable store to track the currently hovered node. Define two derived stores that contain a Set indicating which nodes and links should be highlighted. Hint: look up how sets work in JavaScript! These sets should contain all nodes and all links when there is no currently hovered node. Otherwise, they should contain only the hovered node and its neighbors and the links between them. Add these derived stores to the object that is returned from the function. Finally, use the hovering functionality in _ex_3_network.js, add mouse event listeners to the circles that update the hover state, and set the node's and link's opacity to 0.3 when they are not in the sets.

Loading the data, please wait...

Better data loading

The way that data was loaded in these exercises is not optimal. You have probably noticed that the page jumps around when it is refreshed. This happens for two reasons: (1) the loading-messages do not have the same size as the visualizations, and (2) the visualisations are not pre-rendered by our server.

There are two ways we could solve this issue. The first approach removes the if-block we have used so far. As a result, components that depend on the data are shown regardless of whether the data is available yet. This means that we have deal with the case that our data is not available within those components. They have to update themselves when the data becomes available. You have enough Svelte experience by now to figure out how to do this. However, this approach complicates the code in your components. In addition, it breaks the spirit of the RAII principle: objects should acquire their resources during their initialization and release them when they get destroyed.

The second approach relies on SvelteKit rather than pure Svelte. Unlike Svelte, SvelteKit is able initialize components with pre-loaded values. We only have to tell it how it should load our data! Unfortunately, this only works when a component is shown as a page and not when it is imported and used on a page. So, to see this approach in action, you have to look at the bonus exercise directly. You can also compare it to exercise 1. The difference is most obvious when you refresh both pages a couple of times!

To tell SvelteKit how to load our data, we have to define a module-script in the component: 

<script context="module">
  export async function load({ fetch }) {
    const response = await fetch("/data/starwars.json");

    return {
      status: response.status,
      props: {
        data: response.ok && (await response.json())
      }
    };
  }
</h5script>

This should be an additional script-tag within the component, separate from the ones we have used in components so far! Within that module-script, you can export an asynchronous function called load(). This function is given an object as input, but we only need the fetch property, which we extract using object destructuring. Within the load function, we can use fetch() to load our file. You should not use the d3.js loading functions here because they use the native fetch funcion and not the one given by SvelteKit. Finally, we have to return an object with two values: status and props. The status value comes directly from the response we got from fetch. Within props we can specify values for the component properties that our component should be initialized with. Here, we only need a value for the data array. To get that value, we interpret the text contained in response as json by calling the .json() function on it. Look at the d3-dsv module if you need to interpret text as csv! Now, SvelteKit will instantiate our component with the data property already filled in!

There are no exercises about this section. I just really wanted to show how you can avoid the page-jumping! Perhaps you can use this approach for your projects later in the course :)