Questioning System for Publiclab – Expanded Q & A Project

The GSoC Coding Period has already started and its been a week since then. This post contains the work that I have done during the first week of Coding period.

Here are the list of tasks that I worked on during the first week

1. Create a questions page that lists all the questions asked in the community.
2. Create sections in questions page that shows Recently viewed, Most viewed and Most liked questions. They are shown in Recent, Popular and Liked tab respectively.
3. Create a questions show page that displays the question along with its description.
4. Placing buttons for editing, deleting and liking the question in the show page.
5. Modify post mechanism for questions.
6. Writing unit tests for the implemented changes.

Coming to the technical point of the implementations let me just walk through the Basic structure of the resources used for posting Research Notes in Publiclab.

All Research notes and wiki pages in plots2 codebase  are stored as nodes and accessed by the drupal_node model (Don’t think this has something to do with drupal. They probably migrated from drupal to Rails hence the name). In the current codebase a node has basically three ‘types‘. It is either note, wiki or maps.

plots2 has a tagging system where tag names are stored in a term_data table accessed by the drupal_tag model. There are some tags called power_tags that have a name like something:foo.

Questions are notes that have a question power_tag i.e. the tagname is question:foo. So for listing questions in the question page all I had to do was to list the notes with such tags. I had to create a questions controller for this purpose. The Recent tab listed questions in the descending order of post creation date. The Popular tab listed questions in descending order of the number of views. The Likes tab listed questions in the descending order of the number of likes.

Since there was already an posting mechanism and questions were actually a special kind of notes I had to just reuse the code and eliminate any redundancy. Any piece of software you write must always take care of this. Rails already supports this by writing DRY(Don’t Repeat Yourself) Code. So for post, edit and delete actions I had to modify the notes controller it so that it could work well for questions. Since the work is in the development phase I also had to take care that users don’t accidentally land up on these pages while they try to post a question or edit it that have a question:foo tag. This was taken care off by appending a :redirect parameter in the url of post and edit paths for the new feature. The post request would land up on the questions show page only if there is :redirect parameter in the url set to question.

Another bug that I solved in the way was that previously pages that required logged in access didn’t redirect users to the specific page rather they were redirected to the user dashboard page. I solved it to make it running. This could be used for the Ask a question button that required log in. Once the users are logged in they were taken to the post form.

Finally I had to write thorough test code for all the features implemented. Any codebase that needs to be used in long run must have thorough testing. My mentor Jeff  also supported this fact. Though I had experience of writing tests in Rspec, a Ruby testing framework the plots2 repository used Rails test::unit for testing purposes. So I started learning how to write tests in test::unit. It is really good to see the green dots for passing tests that you write! Once you get a hang of it it’s really fun writing test code! The Rails testing guide has some really good documentation on writing tests.

You can see my full work on this in plots2 Pull Request #550.

Parsing RSS feeds

There are many ways of Parsing RSS feeds data so that it can be used for showing useful information in a html format. Here I will show how we parse rss feed data using JavaScript that can be presented  in a webpage.

RSS is a type of document that uses standard web feed format for publishing frequently updated data like blog posts, headlines, audio or video. It is analogous to xml which just documents data primarily rather than rendering them. RSS feeds have a standard format of representation. You can find the specifications in this link. Here is a sample RSS 2.0 feed that documents map data for an author in mapknitter.org maintained by Publiclab.

To parse the rss feeds I used the jQuery.get() method which is function that handles Ajax requests. We use this method to load the rss feed using a get request

	$.get('https://mapknitter.org/feeds/author/waren', function (feed) {
	// processing code goes here
	})

view raw

jQuery_get.js

hosted with ❤ by GitHub

As you can see in the RSS feed each item is surrounded by an <item> tag and the whole content is wrapped by a <channel> tag. The $.get() function on Ajax success returns the response in the feed variable. The feed variable thus has the content of the RSS feed.

Suppose we want to get the title and description of items in the feed. The processing code will then be

	$.each($(feed).find('channel item'), function (i, item) {
	title = $(item).find('title').html(),
	description = $(item).find('description').html();
	// Rendering code goes here
	})

view raw

rss_snippet_1.js

hosted with ❤ by GitHub

Similarly, we can get other elements of the item. Now we can use this data along with some JavaScript to render it on a webpage.

I had a table within a div with an id ‘maps’ to list all maps fetched from the rss feed data. So what I did was append some rows and cells to it to list them . The rendering code was similar to this

	$('#maps table').append('<tr class="feed-item-' + i '"></tr>');
	var itemEl = $('#maps table .feed-item-' + i);
	itemEl.append('<tr class="title"></tr>');
	itemEl.find('.title').html(title);
	itemEl.append('<tr class="description"></tr>');
	itemEl.find('.description').html(description);

view raw

rss_snippet_2.js

hosted with ❤ by GitHub

So  you see it’s pretty simple Parsing RSS feeds using some JavaScript code. You can see my actual work in commit 4127568.

Creating Hashtags

The GSoC Community Bonding period has already started and I am really excited to be a part of it. This week I got to work on a issue for Creating Hashtagging System. It’s really simple. All you have to do is to find words that are preceded by a # and replace them with suitable links.

For matching keywords preceded by a Hashtag I used a Regular expression, keywords that would match the regular expression was replaced with suitable linking markup. I am working on Rails framework. So I will show here how to write Ruby code for implementing Hashtags. If you are new to Rails you can perhaps see how to setup a development environment in my previous post.

I am defining the Regular expression in a constant defined in the config/initializers/constants.rb file. It is a general convention to define constants that are used by the application in files in config/initializers directory. The regular expression that I used was

HASHTAG = /(\s)\#([\w-]+)/

What the Regular expression means is that, it should match words that has a space before a ‘#’ followed by any word character (i.e. letter,number or underscore) or dash. The constants were defined under a module named Callouts. Ruby modules are like wrappers around some class or methods. So we use the module to access these constants. The next thing is to define the markup that was to be replaced in place of the keywords. So that is also defined in the constants.rb file under the Callouts module. In the present case we had two formats for linking one for use in markdown and one for linking as html markup. So the markup texts goes as follows

HASHLINKMD = '\1[#\2](/tag/\2)'
HASHLINKHTML = '\1<a href="/tag/\2">#\2</a>'

Matches for Hashtags are generated in two groups. as you can see above the HASHTAG has groups covered in round brackets. The first group matches a space and the second group has the keyword. The markup texts uses these groups as ‘\1′ and ‘\2′.

Now we have defined the matcher and replacement markup, we actually need something to make it happen. Here comes the role of gsub. gsub is a method of String class that replaces the all the matches found within the string with the second argument given. The first argument is generally a Regular expression e.g.

"hello".gsub(/[aeiou]/, '*')                  #=> "h*ll*"

So to replace the hashtag with the appropriate markup I used

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKHTML))

where body is a String class object.

Similarly to replace the hashtags with the markdown markup the code will be

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKMD))

I have used the method const_get to fetch the regular expression. const_get finds a constant with given name in the specific module or Class.

Hurray! We have successfully made a Hashtagging system. Now you can show the desired content in the page linked to the Hashtags.

Well there was much more to it. There were many edge cases that I had to test for. My mentor Jeff helped me point it out. You can see them here. as we were dealing wit markdown I had to take care that Headings weren’t linked as Hashtags. The key was choosing a correct Regular expression. You can test Regular expressions in Rubular as I did here. You can see my full work in commits 11e0dd5 and 6bd065a.

Getting Started with Ruby on Rails

Here I will go through a How to setup a Ruby on Rails Project on your local machine along with a development database. The Setup is shown for a debian based package manager.

Installing Ruby

First of all we need to install a stable version of Ruby. You can install Ruby using a number of methods. The full list is given on the ruby-lang website. The most easiest one would be through the package manager. But I would strongly suggest NOT to use that method because it install an old version of Ruby and also gives limited flexibility. The most popular used method is by using Ruby Managers as it allows you to install and manage multiple versions of Ruby and gemsets. You will find different Projects use different versions of Ruby and Ruby Managers makes it easy to cater all needs.

Here I will show how to install  ruby with RVM. First install the mpapis public key using

$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

Then install RVM along with the latest version of Ruby using

$ \curl -sSL https://get.rvm.io | bash -s stable --ruby

To install specific version of ruby use

$ rvm install 2.1.1

After you have installed Ruby run

$ ruby -v

This gives you the current version of Ruby. The output will be similar to

ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-linux]

Installing Rails

Now you have Ruby installed in your local machine. The next step is installing Ruby gems. Ruby gems are a kind a package manager for Ruby that is used for installing Ruby libraries easily from a server that distributes them. For installing a Ruby gem the command is like

$ gem install will_paginate

Rails is one such Ruby gem. To install the latest version of Rails run

$ gem install rails

Installing a Database

Here I will show how to install mysql database and make it running with Rails. For installing mysql server run

$ sudo apt-get install mysql-server

You will be prompted to give a root user password. Enter it and you will see mysql server installing. To use mysql in Rails you need to install the mysql2 gem .For that you also need the libmysqlclient for building the gem. T o install it run

$ sudo apt-get install libmysqlclient-dev

Then install the mysql2 gem using

$ gem install mysql2

You are now ready to setup a Ruby on Rails Project on your local environment.

Installing a Ruby on Rails Project

Though most applications have different installation instructions depending on other dependencies. But most setups follow some general steps. You can find the installation instructions of plots2 here. Let us see the basic steps to get a Ruby on Rails application running on your local server.

First you need to install all the gem dependencies. All gems used in the application are listed in the Gemfile. To install them run

$ bundle install

By running this command bundler installs all gem dependencies in the application. Bundler itself in fact is a Ruby gem. If you don’t have bundler installed install it by using

$ gem install bundler

Now you have all the gems installed. Next you have setup the database connection. To setup the database connection open  config / database.yml. Enter the database username and password. The database.yml looks similar to this

	default: &default
	adapter: mysql2
	encoding: utf8
	pool: 5
	username:
	password:
	socket: /var/run/mysqld/mysqld.sock
	development:
	<<: *default
	database: simple_cms_development
	test:
	<<: *default
	database: simple_cms_test
	production:
	<<: *default
	database: simple_cms_production
	username: simple_cms
	password: <%= ENV['SIMPLE_CMS_DATABASE_PASSWORD'] %>

view raw

database.yml

hosted with ❤ by GitHub

To create the database run command

$ rake db:create

To run all migrations use command

$ rake db:migrate

You are now ready to run the application. Rails ships with WebBrick Server for development. So you might have noticed I didn’t mention any step for installing any server. But you are free to use any other server. You can search the web for integration of Apache or Nginx with rails. To run the server run

$ rails s

Congratulation! You have the server running! Go to localhost:3000 in your browser to see your application running.

Journey Begins with GSoC

So finally after 1 month of waiting the results are out and I got selected for GSoC 2016. I am going to work with Publiclab on its Expanded Q & A System for this summer. You can see my Project proposal here if you like. I have already contributed to plots2 repository on Github and I am really excited to work on my Project soon.

Publiclab.org is a community dealing with Environmental issues and it has developed its own DIY techniques to tackle environmental problems. My job for this summer would be to develop a Q & A forum so that people can raise their questions effectively and make it easier for those in the community to answer them. As a developer community Publiclab is really encouraging and all its work are Open Source on Github. It has some great first-timers bugs worth looking. I would like to thank my mentor Jeff who has specifically been very encouraging and helped a lot to understand the plots2 codebase. Liz and Daniel are my other mentors.

Hoping to have a great summer with Publiclab and GSoC!