Wiki

Information and Resources

User Tools

Site Tools


api:pagination_example

JournalMap API Pagination Example

JournalMap's API uses pagination for some endpoints (articles, authors, locations) to protect the server from very large request. This is a simple example to show how to iterate over the multiple pages of an API result to display a map of point clusters for a selected journal. For this simple example, the map points are attributed only with the article title and the link back to JournalMap for the article. This example uses D3.js, Leaflet, and the Leaflet Marker Cluster plugin. It also uses queue.js to hold processing of the articles until all the pages have been loaded, and spin.js for a nifty spinning wheel icon while the map loads.

Article Locations for Journal of Arid Environments


Change publication:


Steps for creating the example

To see the full script put together, see our example on GitHub.

1. Add the script references to the page header

Add the following references to the <head> section of your webpage:

	<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>
	<script src="http://d3js.org/queue.v1.min.js"></script>
	<script src="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.js"></script>
	<link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.css" />
	<link rel="stylesheet" href="http://leaflet.github.io/Leaflet.markercluster/dist/MarkerCluster.css" />
	<link rel="stylesheet" href="http://leaflet.github.io/Leaflet.markercluster/dist/MarkerCluster.Default.css" />
	<script src="http://leaflet.github.io/Leaflet.markercluster/dist/leaflet.markercluster-src.js"></script>
	<script src="http://fgnass.github.io/spin.js/spin.js"></script>

2. Add a container to hold the map

Put a <div> tag in the body of your page where you want your map to be. Give it an id (e.g., id=“jmap”) and set any other properties you need to. I added a header element with id=“pubname” that we'll modify to show the name of the publication we're showing the heatmap for. Also, I added a <div> tag nested inside my “jmap” <div> for a nifty little spinner to show that the map is loading. Some publications have many pages of results and it takes a few seconds to call all the pages from the API and assemble the result. Finally, I wrapped everything into a master <div> tag to keep it all contained.

  <div id="container" style="pading:8px;width:600px;border:thin solid black;box-shadow: 5px 5px 2px #888888">
    <h3 id="pubName">Article Locations for Journal of Arid Environments</h3>
	<div id="jmap" style="height:400px; width:600px" >
		<div class="spinner" id="jmapSpin"></div>
	</div>
  <br>
  Change publication: <form><select name="pubSel"></select></form>
  </div>  

3. Add a script block and set up the environment

Next we'll add a script block to the body of the HTML and set up some variables and objects that we'll need later on. This script pre-populates the heatmap with the journal that is specified with pubName and pubID. You'll need to supply your own API key too. The spinner object defines the options for the progress spinner wheel. Tweak these as you want.

	var data = []; // a global to store the json response
	var pubName = "Journal of Arid Environments";
	var pubID = 1;
	var centerLat = 0;
	var centerLon = 0;
	var Zoom = 1;
	var apikey = "PjFRefUzFYzFCvshie8Q";
	//var page = 1;
 
	var spinner = new Spinner({
		lines: 12, // The number of lines to draw
		length: 7, // The length of each line
		width: 3, // The line thickness
		radius: 10, // The radius of the inner circle
		color: '#000', // #rbg or #rrggbb
		speed: 1, // Rounds per second
		trail: 100, // Afterglow percentage
		shadow: false // Whether to render a shadow
	})

4. Get the list of publications to populate the dropdown box

The next step is to start digging into the API and getting some data. The next block of code uses D3's JSON method to hit the publication endpoint of the API and request a list of publications. This list is then used to populate the drop-down select box with the journal names (with their article counts) and their ID numbers. Lastly, we turn event listening on for the select box and trigger the pubChanged function when changes in the selected journal happen.

// Get the list of pubs to populate the dropdown box
d3.json("https://www.journalmap.org/api/publications.json?key="+apikey+"&version=1.0", function(error, json) {
	  if (error) return console.warn(error);
 
	  var dropDown = d3.selectAll("select");
    dropDown.selectAll("option")
      .data(json)
      .enter()
        .append("option")
        .attr("value", function (d) { return d.id; })
        .text(function (d) { return (d.name+" ("+d.articles_count+" articles)"); });
    dropDown.on("change",pubChanged);
	  });

5. Initialize the map

Now it's time to set up the Leaflet map that we're going to use. The setup of the map is pretty straightforward from the Leaflet documentation. The only new twist here is that we define a new object for the heatmap layer (we'll add it to the map later on). Finally, we call the function buildMap that will pull the data from the API and create the points for the map.

// Initialize the map
var map = L.map('jmap').setView([centerLat, centerLon], Zoom);
L.tileLayer('http://{s}.tile.thunderforest.com/landscape/{z}/{x}/{y}.png', {
   maxZoom: 18,
}).addTo(map);
var markers = L.markerClusterGroup();
 
// Build the map
buildMap(pubID);

6. Iterate over the pages for the publication and assemble the coordinates for the map points

The buildMap function is where all the work happens in making the map. This function takes as an argument the publication ID number. Initially this is set to 1 (for Journal of Arid Environments), but it gets updated if a different journal is selected from the drop-down box. Here's what's happening in this function:

  1. The first thing this function does is call the articles endpoint of the API for that publication ID to get the number of pages (from the response header) that we'll need to deal with to get all the articles (37 pages for all the articles from Journal of Arid Environments).
  2. Then we set up the spinner so the user knows something is going on while we load all the data.
  3. Next we set up the iteration over the pages with a for loop.
  4. Because JavaScript is asynchronous, we need to use the queue() function (from https://github.com/mbostock/queue) to make the browser wait until all the pages are loaded before going on with the code. If not, then you'll get partial results (or sometimes no results at all).
  5. The output of queue() and awaitAll() is an array of article arrays. We use another for loop to smash all these together into a single long array of the articles.
  6. Next we loop through this article array, pull out the coordinates for the articles, and create map markers (points) for each one. We add the points to the markers layer group.
  7. Once we have the markers added, we can stop the spinner and the layer group to the map!
// Load the first page for the publication
function buildMap (id) {
	d3.xhr("https://www.journalmap.org/api/articles.json?key="+apikey+"&page=1&publication_id="+id+"&version=1.0", function(error,pubjson) {
			var pages = pubjson.getResponseHeader('X-Pages');
			spinner.spin(document.getElementById("jmapSpin"));
			var articles = [];
			q = queue();
			for(page=1;page<=pages;page++) {
				//console.log("page "+page+" of "+pages);
				q.defer(d3.json, "https://www.journalmap.org/api/articles.json?key="+apikey+"&page="+page+"&publication_id="+id+"&version=1.0")
			}
			q.awaitAll(function(error,files) { 
 
				for (f=0;f<files.length;f++) { 
					articles.push.apply(articles,files[f]);
				};
 
				// Assemble the markers for the map
				markers = L.markerClusterGroup();  // Empty out the markers if they exist already
				for (i=0;i<articles.length;i++) {
					var article = articles[i];
					var locations = articles[i].locations;
					for (j=0;j<locations.length;j++) {
						var marker = L.marker([locations[j].latitude,locations[j].longitude]);
						marker.bindPopup("<strong><a href='https://www.journalmap.org/articles/"+article.id+"/' target='_blank'>"+article.title+"</a></strong>"); 
    					markers.addLayer(marker);	
					};
				};
				map.addLayer(markers);
 
				spinner.stop(document.getElementById("jmapSpin"));
 
			});
 
	});
 
};

7. Define the function for responding to changing publications in the dropdown

This last function defines what happens when the selected value from the dropdown box changes. First we get the new selected value (publication ID) and corresponding pub name. We update the header text with the new pub name. We need to remove the existing marker (point) layer and then call buildMap with the new selected publication ID.

function pubChanged() {
	var selectedValue = d3.event.target.value;
	var name = d3.event.target.options[selectedValue-1].text;
	//console.log(d3.event.target.options[selectedValue-1].text);
	map.removeLayer(markers);
	buildMap(selectedValue);
        d3.select("#pubName").text("Article Locations for "+name);
}; 
api/pagination_example.txt · Last modified: 2015/01/21 08:46 by jkarl

FEEDBACK    DOWNLOADS    ABOUT    SUPPORT US    HELP    BLOG    WIKI

JournalMap data is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
Creative Commons License