Commenting System for Publiclab – Expanded Q & A Project

The fourth week of GSoC Coding Period is also over and the Mid-term evaluations are just about to come. So coming Mid-way of the GSoC journey, here are the final tasks that I completed before the Pre-midterm period.

  1. Modify existing comment system to handle answer comments and question comments
  2. Create new views and styles for comments in the Q & A pages
  3. Create expandable text boxes for comments and a View more functionality for comments

These are the brief overview of the features implemented. Let me now come to the technical details.

The current commenting system for Research notes uses a DrupalComment model for handling the comments in the database. So I used the same model for answer comments too. For answer comments I just had to introduce a aid column in the comments table and relate comments to answers. But I had to take care of a thing that answer comments weren’t related to the question otherwise all the answer comments would also show up in the question comments section in the page. So what I did was store a 0 in the nid field of answer comments to unrelate any answer comments to questions.

I also used the same comment controller for hadling requests for answer comments with some modifications. Took me to create new js partials for answer comments as comments were created and deleted witj Ajax requests.

The next work that I completed was introduce a new comments view for the Q & A pages. One of the new feature was creating a View more button for questions so that more comments were shown on pressing the View more button. Comments are listed in the descending order of their creation. So recent comments are listed at the top and older comments are shown as you Click View more.

Another cool feature that I implemented was expanding text boxes for comments. Did you ever notice how text boxes of facebook comments expand as you type in more lines without giving a annoying scrollbar. It’s the same feature that I implemented. I followed this article for that. I just used jQuery instead of raw JavaScript. It’s a really good feature when you want to save space as well give a good User Experience.

That’s all before the Mid-term evaluations. You can find this work in plots2 Pull request #589

Answering System for Publiclab – Expanded Q & A Project

Going over the third week in GSoC Coding period, a lot of Progress has been made over this week. A large chunk of Code has been merged and it makes me relieved that an important part of the Project is complete. This week I developed the Answering system for the Q & A Project which is the next most important part after the Questioning system.

Here is the summary of the tasks that I covered in the Answering system

  1. Create a new Answer model for handling Answers to each question
  2. Develop Post, edit and delete methods for Answers
  3. Design how Answers are listed for each Question in the Question page
  4. Create Like functionality for Answers
  5. Write functional and unit tests for the features implemented

These are the overview of the features implemented. Let me now go through the technical part of the implementations.

First is the Creation of the new Answer Model. As already mentioned in earlier posts all content in plots2 is handle by the DrupalNode model. Questions are a just type of node. Each question can have any answers so the Answer model is related to the DrupalNode model and also to a DrupalUser model that is the model which handles users in plots2 since an answer is written by an user. After setting up the relation, validations and various methods for the model had to be written for the model. Validations refer to the set of requirements that each record in the table must follow like the content of an answer can’t be blank. Rails defines a set of Validation methods for a model that handles this.

Next was creating a answers controller for handling the Post, edit and delete mechanisms for the answers. Answers are posted using Ajax requests in plots2. So is the delete action. Update action uses in-place edits for editing answers but the Update action is a normal http request. Working with Ajax in Rails is as simple as anything. Rails handles Ajax requests using non-obstrusive JavaScript. You just have to define a data-remote property in the links or forms for making Ajax requests and Rails will smartly handle each Ajax call. You can read more about it here.

Now coming to the Like functionality for questions. The Like button also involves Ajax requests. It is basically a button that Likes or unlikes an answer on button press. To implement the Like functionality a new AnswerSelection model had to be introduced. Eack like is related to the answer and the user who liked it. The answer_selection table has a liking field that takes a boolen value. It is set to true when an answer is liked and it is set to false if the answer is unliked. To deal with the Ajax calls I created a new answer_like controller which has a likes action that updates the liking field of the record.

Finally writing unit tests for the new models and functional tests for the new controller and implemented changes is a required part of good code.

On top of that I got to had a nice conversation with David Days and Ujitha of the Advanced search Team and moved the question Search functionality to a separate controller to avoid any conflicts and better collaborate with each other.

You can see the work on Answering functionality in plots2 Pull request #566.

Creating a Basic Search functionality

The second week of GSoC coding period is over and here are the works that I completed during this week.

This week was mostly spent on improving the design of the Questions page as proposed by my mentor and introducing a Basic question search functionality. I had to do some follow up fixes for my previous changes like adding comments for some temporary changes and one most important fix was to have a custom method defined for questions url so that it could be used at places required instead of writing out the full path every time.

Following the design proposed in plots2 issue #554 I modified the design for the questions page using some of the dashboard templates used in plots2.

You can now see the new questions page at https://publiclab.org/questions/ .

As proposed in the design I made a Basic Search functionality for Questions. Here I will walk through How I made the Basic Search functionality for questions.

There was already a search functionality that searched for notes, wikis, maps and comments from the  search box in the navbar. Autocomplete results are shown as you type in any keywords showing a icon and title in the autocomplete results suitably linked with the item. There is also an advanced search which you can see in the url  https://publiclab.org/search/ . There you can search selectively for any research note, wiki, map or comment by checking the checkboxes present there.

The question search is similar to the search except that it searches only questions. The actions for the questions search would go in the search controller. So the search query searches for any notes whose title matches the keyword and also has a question:foo tag. The search query for that is


@notes = DrupalNode.where(
'type = "note" AND node.status = 1 AND title LIKE ?',
"%" + params[:id] + "%"
)
.joins(:drupal_tag)
.where('term_data.name LIKE ?', 'question:%')
.order('node.nid DESC')
.page(params[:page])

view raw

search_query.rb

hosted with ❤ by GitHub

Here the :id is the search keyword typed. You can see my previous post for the definition of nodes and tags used in the codebase.

Now that I have fetched the questions I have to implement the autocomplete feature. I did it using the jQuery typeahead function, similar to what was done before for the search feature.

The JavaScript code for typeahead goes as follows


$('#questions_searchform_input').typeahead({
items: 15,
minLength: 3,
source: function (query, process) {
return $.post('/questions_search/typeahead/' + query, {}, function (data) {
return process(data);
})
},
updater: function(item) {
var url;
if ($(item)[0] != undefined) url = $(item)[0].attributes['data-url'].value;
else url = '/questions_search/' + $('#questions_searchform_input').val();
window.location = url;
}
})

I have a input field with an id ‘questions_searchform_input’. The items option tells the maximum items that can be listed in the results. The minLength option specifies the minimum characters that will trigger the search. The source option gives the dataset of search results. Here we have used a ajax post request that submits the keyword to the /search/questions_typeahead/:id  url.

The actual work of fetching the results is done in the search controller. It has two actions questions and questions_typeahead. The questions action shows the results searched by the keyword and displays them on submit. If no match is found it redirects the user to the post form to post  new question. The questions_typeahead action generates the results for the autocomplete results.

You can find my work on the search functionality in the commit ceabfb0.

Lastly I wrote some functional tests for the search controller and also an integration test to test the entire search mechanism. You can find my full work in the plots2 Pull Request #555

This is just a simple search functionality. There is much more to it for developing a advanced search functionality. In fact there is a parallel Advanced Search Project going on in Publiclab lead by David Days. In course of time the search code will be modified to meet the advanced search needs.

 

Questioning System for Publiclab – Expanded Q & A Project

The GSoC Coding Period has already started and its been a week since then. This post contains the work that I have done during the first week of Coding period.

Here are the list of tasks that I worked on during the first week

  1. Create a questions page that lists all the questions asked in the community.
  2. Create sections in questions page that shows Recently viewed, Most viewed and Most liked questions. They are shown in Recent, Popular and Liked tab respectively.
  3. Create a questions show page that displays the question along with its description.
  4. Placing buttons for editing, deleting and liking the question in the show page.
  5. Modify post mechanism for questions.
  6. Writing unit tests for the implemented changes.

Coming to the technical point of the implementations let me just walk through the Basic structure of the resources used for posting Research Notes in Publiclab.

All Research notes and wiki pages in plots2 codebase  are stored as nodes and accessed by the drupal_node model (Don’t think this has something to do with drupal. They probably migrated from drupal to Rails hence the name). In the current codebase a node has basically three ‘types‘. It is either note, wiki or maps.

plots2 has a tagging system where tag names are stored in a term_data table accessed by the drupal_tag model. There are some tags called power_tags that have a name like something:foo.

Questions are notes that have a question power_tag i.e. the tagname is question:foo. So for listing questions in the question page all I had to do was to list the notes with such tags. I had to create a questions controller for this purpose. The Recent tab listed questions in the descending order of post creation date. The Popular tab listed questions in descending order of the number of views. The Likes tab listed questions in the descending order of the number of likes.

Since there was already an posting mechanism and questions were actually a special kind of notes I had to just reuse the code and eliminate any redundancy. Any piece of software you write must always take care of this. Rails already supports this by writing DRY(Don’t Repeat Yourself) Code. So for post, edit and delete actions I had to modify the notes controller it so that it could work well for questions. Since the work is in the development phase I also had to take care that users don’t accidentally land up on these pages while they try to post a question or edit it that have a question:foo tag. This was taken care off by appending a :redirect parameter in the url of post and edit paths for the new feature. The post request would land up on the questions show page only if there is :redirect parameter in the url set to question.

Another bug that I solved in the way was that previously pages that required logged in access didn’t redirect users to the specific page rather they were redirected to the user dashboard page. I solved it to make it running. This could be used for the Ask a question button that required log in. Once the users are logged in they were taken to the post form.

Finally I had to write thorough test code for all the features implemented. Any codebase that needs to be used in long run must have thorough testing. My mentor Jeff  also supported this fact. Though I had experience of writing tests in Rspec, a Ruby testing framework the plots2 repository used Rails test::unit for testing purposes. So I started learning how to write tests in test::unit. It is really good to see the green dots for passing tests that you write! Once you get a hang of it it’s really fun writing test code! The Rails testing guide has some really good documentation on writing tests.

You can see my full work on this in plots2 Pull Request #550.

Parsing RSS feeds

There are many ways of Parsing RSS feeds data so that it can be used for showing useful information in a html format. Here I will show how we parse rss feed data using JavaScript that can be presented  in a webpage.

RSS is a type of document that uses standard web feed format for publishing frequently updated data like blog posts, headlines, audio or video. It is analogous to xml which just documents data primarily rather than rendering them. RSS feeds have a standard format of representation. You can find the specifications in this link. Here is a sample RSS 2.0 feed that documents map data for an author in mapknitter.org maintained by Publiclab.

To parse the rss feeds I used the jQuery.get() method which is function that handles Ajax requests. We use this method to load the rss feed using a get request


$.get('https://mapknitter.org/feeds/author/waren', function (feed) {
// processing code goes here
})

view raw

jQuery_get.js

hosted with ❤ by GitHub

As you can see in the RSS feed each item is surrounded by an <item> tag and the whole content is wrapped by a <channel> tag. The $.get() function on Ajax success returns the response in the feed variable. The feed variable thus has the content of the RSS feed.

Suppose we want to get the title and description of items in the feed. The processing code will then be


$.each($(feed).find('channel item'), function (i, item) {
title = $(item).find('title').html(),
description = $(item).find('description').html();
// Rendering code goes here
})

Similarly, we can get other elements of the item. Now we can use this data along with some JavaScript to render it on a webpage.

I had a table within a div with an id ‘maps’ to list all maps fetched from the rss feed data. So what I did was append some rows and cells to it to list them . The rendering code was similar to this


$('#maps table').append('<tr class="feed-item-' + i '"></tr>');
var itemEl = $('#maps table .feed-item-' + i);
itemEl.append('<tr class="title"></tr>');
itemEl.find('.title').html(title);
itemEl.append('<tr class="description"></tr>');
itemEl.find('.description').html(description);

So  you see it’s pretty simple Parsing RSS feeds using some JavaScript code. You can see my actual work in commit 4127568.

Creating Hashtags

The GSoC Community Bonding period has already started and I am really excited to be a part of it. This week I got to work on a issue for Creating Hashtagging System. It’s really simple. All you have to do is to find words that are preceded by a and replace them with suitable links.

For matching keywords preceded by a Hashtag I used a Regular expression, keywords that would match the regular expression was replaced with suitable linking markup. I am working on Rails framework. So I will show here how to write Ruby code for implementing Hashtags. If you are new to Rails you can perhaps see how to setup a development environment in my previous post.

I am defining the Regular expression in a constant defined in the config/initializers/constants.rb file. It is a general convention to define constants that are used by the application in files in config/initializers directory. The regular expression that I used was

HASHTAG = /(\s)\#([\w-]+)/

What the Regular expression means is that, it should match words that has a space before a ‘#’ followed by any word character (i.e. letter,number or underscore) or dash. The constants were defined under a module named Callouts. Ruby modules are like wrappers around some class or methods. So we use the module to access these constants. The next thing is to define the markup that was to be replaced in place of the keywords. So that is also defined in the constants.rb file under the Callouts module. In the present case we had two formats for linking one for use in markdown and one for linking as html markup. So the markup texts goes as follows

HASHLINKMD = '\1[#\2](/tag/\2)'
HASHLINKHTML = '\1<a href="/tag/\2">#\2</a>'

Matches for Hashtags are generated in two groups. as you can see above the HASHTAG has groups covered in round brackets. The first group matches a space and the second group has the keyword. The markup texts uses these groups as ‘\1′ and ‘\2′.

Now we have defined the matcher and replacement markup, we actually need something to make it happen. Here comes the role of gsub. gsub is a method of String class that replaces the all the matches found within the string with the second argument given. The first argument is generally a Regular expression e.g.

"hello".gsub(/[aeiou]/, '*')                  #=> "h*ll*"

So to replace the hashtag with the appropriate markup I used

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKHTML))

where body is a String class object.

Similarly to replace the hashtags with the markdown markup the code will be

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKMD))

I have used the method const_get to fetch the regular expression. const_get finds a constant with given name in the specific module or Class.

Hurray! We have successfully made a Hashtagging system. Now you can show the desired content in the page linked to the Hashtags.

Well there was much more to it. There were many edge cases that I had to test for. My mentor Jeff helped me point it out. You can see them here. as we were dealing wit markdown I had to take care that Headings weren’t linked as Hashtags. The key was choosing a correct Regular expression. You can test Regular expressions in Rubular as I did here. You can see my full work in commits 11e0dd5 and 6bd065a.

Journey Begins with GSoC

So finally after 1 month of waiting the results are out and I got selected for GSoC 2016. I am going to work with Publiclab on its Expanded Q & A System for this summer. You can see my Project proposal here if you like. I have already contributed to plots2 repository on Github and I am really excited to work on my Project soon.

Publiclab.org is a community dealing with Environmental issues and it has developed its own DIY techniques to tackle environmental problems. My job for this summer would be to develop a Q & A forum so that people can raise their questions effectively and make it easier for those in the community to answer them. As a developer community Publiclab is really encouraging and all its work are Open Source on Github. It has some great first-timers bugs worth looking. I would like to thank my mentor Jeff who has specifically been very encouraging and helped a lot to understand the plots2 codebase. Liz and Daniel are my other mentors.

Hoping to have a great summer with Publiclab and GSoC!