Drupal Views and VIDI


It’s been a while since I last blogged about this project. We’ve had a very busy semester. Along with our SXSW project (sxtxstate.com), we began working on our Telling Stories with Data project. We had three incredible guests, and I will post video of those events soon. We continued learning Drupal and exploring the VIDI modules that were developed by the Jefferson Institute. And we are now developing out our site, which will be called Texas State of Change, doing our reporting and working with the data associated with Texas State’s status as Hispanic Serving Institution.

To continue with our Drupal training, I did create a handout on Drupal Basics that covers most of what I discussed in my last post. Now, I want to go into a little more detail on exactly how we plan to implement the VIDI modules.

I have a handout that explains how to install and use the modules. We are using the Drupal 7 versions. VIDI was originally created for Drupal 6, and the modules in Drupal 7 have not yet been completely updated and tested. The main VIDI module, which makes uploading of data very simple, has not been updated, and that is mostly due to the Table Wizard module not being ported over to Drupal 7. But there are basic workarounds that I discovered as I began learning how to use the modules. The VIDI modules themselves rely on the Views module, so you need to install Views and any related dependency modules. You also need to install all the VIDI modules individually. Find them at http://drupal.org/project/vidi. Make sure to install all the D7 versions, even if they are beta or not recommended, if you are using D7. The Timeline Map module also uses the Date API module.

To get a sense of how the modules worked, we visited the dataviz.org site  (look under Play) and played around with the modules and test data that was provided. Not all of the modules worked in test (specifically the Motion Graphic), but the rest seem straight forward. The Comparative Data section has most of the charts based on the Google Charts Module.  It explains exactly the kind of data that must be used for each chart type. We may end up using some of the modules on the dataviz.org site and embedding the charts from there, instead of installing our own modules, depending on the efficiency.

In experimenting with the modules, I have learned that they are not a very simple install. The first issue is in uploading data to the database. As mentioned above, Drupal 6 had a Table Wizard module that allowed for easy upload of a csv file to be used with the VIDI modules. Instead, we had to use the Feeds Module (also has to be installed with all its dependencies).  The process is a bit complicated and is explained in the handout, but you have to create a Content Type that has the same fields as your csv (this is a simple text file that holds your data. Fields are separated by commas, each record is on a different line). Then you use the Feeds Importer (under Structure menu) and set up a Feed. Once you set up the Importer, you go to the Feed Importer page, and you can upload the csv file with your data in it.  You can then go under Content and see the data you have uploaded.

Now, you may have an extensive data set that you need to manipulate in Excel before you make your csv. You will need to do whatever analysis and aggregation in Excel and then export a csv that has only what you want to include in the chart.

Now that you have your data in your Drupal database, you can set up a View and use one of the VIDI modules as the Format. I discussed the Views module briefly in my previous post, but it is quite powerful in how it aggregates and presents data. You can set up a View to be a page or a block. When you begin to edit the View, you can add the Fields you would like to use in your View. Then you can use Settings under Format to allow you to modify the chart. Play with it. It takes a little getting used to. When you use the Settings, you can specify to add an Embed link that will allow you to embed the resulting chart in a blog post. We are also considering using the Block that was created in lieu of embedding the chart.

I encountered some problems in using these modules. First, there isn’t much in terms of documentation, thus this contribution to the knowledge about the VIDI modules. But I came to realize that some of the problems I was having had to do with my host. It seems that my Bluehost account sometimes runs out of memory when creating Views, thus I get the Page Not Found error. But only sometimes. Very frustrating. I had to seek the advice of some local Drupal experts to make sure that I wasn’t doing something wrong myself. One workaround is to develop the Views on a local install of Drupal (on your desktop), so that the memory limitations of your host don't come into play when creating your Views.

So, that’s it for the VIDI modules. We have several options with which we are experimenting and we have also covered a few other tools that we will ultimately integrate into our project, including Google Fusion Tables and the Google Charts API. We also did PHP and Ruby on Rails exercises, but I don’t think we will be using those tools for the final project. It is important, however, for students in this Advanced class to get a broad exposure to programming. I hope that they will find this broad, yet brief, introduction to these tools to be valuable to them.

My next several posts will have to do with the speaker series we developed this semester:

Aron Pilhofer, New York Times, March 8

Thomas Levine, Scraperwiki, March 21

Dante Chinni, Jefferson Institute, April 11