GSoC 2016 Wrap-up:The End of a wonderful Journey

So GSoC 2016 comes to an end as the thirteenth week in the Coding period wraps up! Well actually it is way beyond these 13 weeks when it started. It was actually in the beginning of March when I started interacting with PublicLab. It’s been a wonderful experience working so far. The place where my actual Open-source journey began. And I have learnt so much in the way.

Final Works

Here is my final report that I will submit for the Final evaluation:

GSoC 2016: Final Work Product of Expanded Q & A System for publiclab.org

This research note contains the detailed report of my work and  the contributions I made.

I also want to show my contribution graph here

contribution

By the way you can find me on Github with my username @ananyo2012

The design changes are merged and I also managed to make some contribution to the Rich Editor. Here is my PR #40 in the PublicLab.Editor repo though it is not merged yet. I really need to learn nodejs before I make any significant contributions to the Rich Editor. Also I made an Rich Editor update in plots2 in PR #664. I spent the week doing fixes on small and some large bugs a creeped up. Also I wrote a wiki on Q & A system and made the final research note for my evaluation.

Also as my mentor insisted I made some first-timer issues in plots2 that could be taken up by new contributors who are completely new to open source. This was a great move to welcome new contributors to our codebase and applause to PublicLab for doing such a great job!

And finally it was all set to go for the Final Evaluation! Results would come by August 30th!

Tough times and Lesson learnt

There were some breaking changes after the code got deployed. It was due to the PR #600 I made. I started working on this long ago and I hadn’t mentioned about it in my any of my blog posts since I wasn’t sure about how it would turn up. It was a work on updating the slugs for research notes and wikis using friendly_id. But things were tough right from the beginning since there was already a diverse slugging system present and I had to make changes keeping the format of the slugs intact. On deploying the code the old slugs of the notes and wikis got updated to new slugs and the older ones were no longer available. So all previous links failed that pointed to those research notes. The new slugs were supposed to redirect to new ones as done using friendly_id. It used a friendly_id_slug table to store the old urls but unfortunately due to some issues they weren’t saved as expected, also I missed few test cases which didn’t predict this case while testing and it was a complete disaster. But fortunately PublicLab had good database backups and things were reverted back to normal in no time.  Some code from the PR had to reverted back and the issue was fixed.

Moral of the story:  Always write good tests and think of rigorous test cases before deploying any code. In fact good tests are the lifeline of a good software development cycle.

Experiences and Best Moments

Well it was an really good experience overall! I really learnt a lot throughout the entire summer. The idea of working alongside with so many people even when you are distances apart is really amazing! We had a Video call with our mentors and other GSoC students at the end of last month and it was great! And people in PublicLab were very helpful giving us reviews in research notes alongside with our mentors.

Thanks to PublicLab and Google OSPO for giving me this wonderful opportunity! Hope to participate in GSoC 2017 again!

Modified Views for publiclab.org – Expanded Q & A Project

The end of the eleventh week in GSoC 2016 and when I look back I am amazed to see the amount of contribution that I made to plots2. For the last couple of weeks I  have been working on designing the interface for some pages, mainly the changes due to the Q & A system. Here are what I have been working on till now

  1. Add a Recently Answered tab in questions landing page that lists out the recently answered questions
  2. Add Q & A to user Profile that would list the questions asked and answered by any user
  3. Create a a distinct sidebar for questions
  4. Add a tag based sort functionality for questions. This would enable filtering questions based on tags
  5. Add a separate question tab in tags page. Tags page contained research notes, wiki and maps earlier
  6. Make it easier to search and ask questions from the questions page by improving the Search/Ask question field that I made earlier.
  7. Finally add links for Questions page in the website header and also put links to Question page and Ask question page in various pages like the dashboard, tags page etc.

This is going to be long PR and I am still working on it; it is nearly completed. Just some little design changes and modifications are needed.

Apart from these there were some important issues that I had to take care while making these changes. I had to distinguish between research notes and questions since questions in plots2 were actually notes marked with a question:topic power tag. So I had to list out research note and questions separately. I made two methods .research_notes() and .questions() in the DrupalNode model that would extract research_notes and questions separately.

Apart from these there were many small design changes that I had to make alongside. Here are some of the screenshots of the pages. They are likely to  be changed in future.

Here is how the questions page looks now when you go to the /questions url

questions

Here is how the content of the questions section in profile page would look. You can see this in the /profile/:username url

user_profile

And here is how the questions will be listed in the /tags/:tagname url

tags

You can find my ongoing work on plots2 PR #628

 

Creating Custom Rake Tasks

As I write this the tenth week in GSoC 2016 is over and it’s the end of the month now. I am a little late with my timeline mostly because I am doing work on the web interface for some pages and things need  to be changed over and over to fit the design choices. But in the mean time I got to learn something new. We had to run two separate rake tasks for running rails tests and JavaScript tests. So there was an issue to run all tasks with one rake task for ease. This is where I learnt about creating custom Rake tasks.

Rake is a Make-like program implemented in Ruby. It comes as a Rubygem and has to be included in your Gemfile during development. Custom Rake tasks start with a namespace followed by the task name separated by a colon. They reside in the lib/tasks directory of a rails app and they go in a file with a .rake extension.

Here is the railsguides documentation on How to create  Custom Rake tasks. Here is this good blog post by ANDREY KOLESHKO while I learnt about writing rake tasks. As you can see there you can either directly write the task in the namespace.rake (remeber to name the file same as the namespace ) file in the lib/tasksdirectory or you can use the rails generator to create the task file

$ rails g task my_namespace my_task

This will create a lib/tasks/my_namespace.rake file where you can write your task.

I needed to run the two tasks rake test and rake spec:javascript from this task. So basically I had to run rake tasks from within a rake task. Here is a stackoverflow answer that answers it perfectly! Know the difference between execute, invoke and reenable.

Here is a custom task I wrote. It can run using rake test:all


# rake test:all
namespace :test do
desc "Run rails and jasmine tests"
task :all do
puts "Running Rails tests"
Rake::Task["test"].execute
puts "Running jasmine tests headlessly"
Rake::Task["spec:javascript"].execute
end
end

view raw

test.rake

hosted with ❤ by GitHub

You can find this in the commit e90fe of plots2. I would be probably posting about the work on design changes in my next post.

Keeping Rails models DRY with concerns

Its over the ninth week in GSoC 2016 and here I am with another of my blog post. Today I will discuss about a pretty nice thing that I learnt while I was working with the notification system. It is related to writing DRY code by using of modules.

My mentor Jeff said that methods in some models were becoming repetitive and they were doing the same kind of task. So it would be better if we could keep the code DRY i.e. avoid redundancy in code. DRY stands for Don’t Repeat Yourself i.e. try to avoid repetitive code in a application. We can do this by  creating methods and modules. Ruby has an awesome concept of modules and mixins  for this purpose.

ActiveSupport Concerns follow the same principle  of Ruby modules and by extending them on Rails models we can include common methods that are shareable across various models.

We had two models named Node and  Answer models. A Node basically served as a container of posts like wikis, questions and research notes in publiclab.org. Answer as the name suggests are answers to questions. Both have some common methods named liked and liked_by. So we made a NodeShared concern and put those in them. Here is how the concern looks like


module NodeShared
extend ActiveSupport::Concern
def likes
self.cached_likes
end
def liked_by(uid)
self.likers.collect(&:uid).include?(uid)
end
end

view raw

node_shared.rb

hosted with ❤ by GitHub

This file goes in a separate app/models/concerns/directory. This is where all concerns reside. To define it as a concern you need to extend the ActiveSupport::Concernmodule as I did above. After that after you have defined the concern you can use it in others models by including the concern like include NodeShared in this case. Here is how the node model would look


class Node < ActiveRecord::Base
include NodeShared # This includes the NodeShared Concern
attr_accessible :title, :uid, :status, :type, :vid, :cached_likes, :comment, :path, :slug
self.table_name = 'node'
self.primary_key = 'nid'
has_many :node_selections, :foreign_key => :nid
has_many :answers, :foreign_key => :nid
def latest
self.revisions
.where(status: 1)
.first
end
def revisions
self.drupal_node_revision
.order("timestamp DESC")
end
end

view raw

node.rb

hosted with ❤ by GitHub

This is a small part of the actual model used. So now the methods in the concern become a part of the model and they can be called with an instance of Node model. Now suppose the Answer model also needs the similar methods. We can simply add this concern to the model as well instead of writing the methods over again.

DHH the creator of Ruby on Rails explained this in his blog post.

Coming back to my progress in Summer of Code I have started working on UI changes and modified views that will include some good features. I will discuss them in my future posts. You can look at my PR #628 though it is Work in progress now. As the mid of July is over its probably going to be more work in the next couple of weeks as things are nearing the end.

Email notifications using Rails ActionMailer

The eighth week of GSoC 2016 coding period is over and I got to work on something new in Rails. This time I was making the email notification system for the Q & A system so that when a user posts a question or answer related people get notified via emails.

Rails has has email delivery framework called ActionMailer. Though I was familiar with Rails, I haven’t worked on ActionMailer before this. So the first thing to do was checking the ActionMailer documentation. I strongly recommend following this  documentation to  understand the email delivery system. Rails guides have good documentation for each version of Rails and the features well explained. It’s a good place to start with.

I had to create a new AnswerMailer and modify the existing CommentMailer and SubscriptionMailer to send out email notifications to users.

A Mailer basically contains some methods that defines the recipient and the subject of the mail to be sent along with the delivery method. It inherits from ApplicationMailer which basically contains the default sender mail address. The mailer view template goes under the views directory which is the standard template directory for Rails. The view template is named according to the mailer method. So for AnswerMailer if there is a method named notify_question_author the template would go with the name notify_question_author.html.erb in the views/answer_mailer directory. Whatever content you want to send in the email goes in this email. So you see the naming convention and functioning quite is similar to Rails controller though not the same.

To send a mail you need to call the mailer method from the controller as described here. Now you need to send mail on specific conditions. Like when you create a post, for example, you would want the subscribers to get specific content while when you post an answer to a question you would like to notify the question followers as well who asked the question. So it is good to create a class or an instance method in related model to do so. Like I defined a instance method answer_notify() for Answer model that notifies the question author as well  as users who liked the question. This contained  two mailer methods AnswerMailer.notify_question_author() and AnswerMailer.notify_answer_likers() which sent emails on meeting some specific conditions. So this way you can frame a custom method that sends the mail and suits the conditions as well keep the code in the controller simple an let the Model contain the method description.

You can find my entire work on this in plots2 PR #612.

GSoC 2016: The Next Phase

I finally made it through the Mid-term evaluations of GSoC 2016! It makes me happy to have made it this far but it’s still not over yet. Moving on to the next phase of GSoC. This ones going to be important as it is longer than the first phase and also I have some pretty good features that I had planned out in my timeline. Here is what I have been working in the past week after the Mid-term evaluations. But before I go to that I want to show you what my mentors gave me as feedback during my Midterm evaluation.

midterm

Really some praising words! Its things like these that really motivates to go on with my work. And that’s an essence of open source too! If you are working with the right community, people really help you out and you get the word of appreciation.

The next feature I have been  working on is making an Accept button for answers. This makes it possible for the author the question to mark an answer as accepted. The “accepted” answer would be marked with a green label  when it would be accepted. The question author can also unaccept it by pressing the same button again. Here is the screenshot of how it would look.

accept_answer

The complete work can be found in the plots2 PR #598

 

Commenting System for Publiclab – Expanded Q & A Project

The fourth week of GSoC Coding Period is also over and the Mid-term evaluations are just about to come. So coming Mid-way of the GSoC journey, here are the final tasks that I completed before the Pre-midterm period.

  1. Modify existing comment system to handle answer comments and question comments
  2. Create new views and styles for comments in the Q & A pages
  3. Create expandable text boxes for comments and a View more functionality for comments

These are the brief overview of the features implemented. Let me now come to the technical details.

The current commenting system for Research notes uses a DrupalComment model for handling the comments in the database. So I used the same model for answer comments too. For answer comments I just had to introduce a aid column in the comments table and relate comments to answers. But I had to take care of a thing that answer comments weren’t related to the question otherwise all the answer comments would also show up in the question comments section in the page. So what I did was store a 0 in the nid field of answer comments to unrelate any answer comments to questions.

I also used the same comment controller for hadling requests for answer comments with some modifications. Took me to create new js partials for answer comments as comments were created and deleted witj Ajax requests.

The next work that I completed was introduce a new comments view for the Q & A pages. One of the new feature was creating a View more button for questions so that more comments were shown on pressing the View more button. Comments are listed in the descending order of their creation. So recent comments are listed at the top and older comments are shown as you Click View more.

Another cool feature that I implemented was expanding text boxes for comments. Did you ever notice how text boxes of facebook comments expand as you type in more lines without giving a annoying scrollbar. It’s the same feature that I implemented. I followed this article for that. I just used jQuery instead of raw JavaScript. It’s a really good feature when you want to save space as well give a good User Experience.

That’s all before the Mid-term evaluations. You can find this work in plots2 Pull request #589

Answering System for Publiclab – Expanded Q & A Project

Going over the third week in GSoC Coding period, a lot of Progress has been made over this week. A large chunk of Code has been merged and it makes me relieved that an important part of the Project is complete. This week I developed the Answering system for the Q & A Project which is the next most important part after the Questioning system.

Here is the summary of the tasks that I covered in the Answering system

  1. Create a new Answer model for handling Answers to each question
  2. Develop Post, edit and delete methods for Answers
  3. Design how Answers are listed for each Question in the Question page
  4. Create Like functionality for Answers
  5. Write functional and unit tests for the features implemented

These are the overview of the features implemented. Let me now go through the technical part of the implementations.

First is the Creation of the new Answer Model. As already mentioned in earlier posts all content in plots2 is handle by the DrupalNode model. Questions are a just type of node. Each question can have any answers so the Answer model is related to the DrupalNode model and also to a DrupalUser model that is the model which handles users in plots2 since an answer is written by an user. After setting up the relation, validations and various methods for the model had to be written for the model. Validations refer to the set of requirements that each record in the table must follow like the content of an answer can’t be blank. Rails defines a set of Validation methods for a model that handles this.

Next was creating a answers controller for handling the Post, edit and delete mechanisms for the answers. Answers are posted using Ajax requests in plots2. So is the delete action. Update action uses in-place edits for editing answers but the Update action is a normal http request. Working with Ajax in Rails is as simple as anything. Rails handles Ajax requests using non-obstrusive JavaScript. You just have to define a data-remote property in the links or forms for making Ajax requests and Rails will smartly handle each Ajax call. You can read more about it here.

Now coming to the Like functionality for questions. The Like button also involves Ajax requests. It is basically a button that Likes or unlikes an answer on button press. To implement the Like functionality a new AnswerSelection model had to be introduced. Eack like is related to the answer and the user who liked it. The answer_selection table has a liking field that takes a boolen value. It is set to true when an answer is liked and it is set to false if the answer is unliked. To deal with the Ajax calls I created a new answer_like controller which has a likes action that updates the liking field of the record.

Finally writing unit tests for the new models and functional tests for the new controller and implemented changes is a required part of good code.

On top of that I got to had a nice conversation with David Days and Ujitha of the Advanced search Team and moved the question Search functionality to a separate controller to avoid any conflicts and better collaborate with each other.

You can see the work on Answering functionality in plots2 Pull request #566.

Creating a Basic Search functionality

The second week of GSoC coding period is over and here are the works that I completed during this week.

This week was mostly spent on improving the design of the Questions page as proposed by my mentor and introducing a Basic question search functionality. I had to do some follow up fixes for my previous changes like adding comments for some temporary changes and one most important fix was to have a custom method defined for questions url so that it could be used at places required instead of writing out the full path every time.

Following the design proposed in plots2 issue #554 I modified the design for the questions page using some of the dashboard templates used in plots2.

You can now see the new questions page at https://publiclab.org/questions/ .

As proposed in the design I made a Basic Search functionality for Questions. Here I will walk through How I made the Basic Search functionality for questions.

There was already a search functionality that searched for notes, wikis, maps and comments from the  search box in the navbar. Autocomplete results are shown as you type in any keywords showing a icon and title in the autocomplete results suitably linked with the item. There is also an advanced search which you can see in the url  https://publiclab.org/search/ . There you can search selectively for any research note, wiki, map or comment by checking the checkboxes present there.

The question search is similar to the search except that it searches only questions. The actions for the questions search would go in the search controller. So the search query searches for any notes whose title matches the keyword and also has a question:foo tag. The search query for that is


@notes = DrupalNode.where(
'type = "note" AND node.status = 1 AND title LIKE ?',
"%" + params[:id] + "%"
)
.joins(:drupal_tag)
.where('term_data.name LIKE ?', 'question:%')
.order('node.nid DESC')
.page(params[:page])

view raw

search_query.rb

hosted with ❤ by GitHub

Here the :id is the search keyword typed. You can see my previous post for the definition of nodes and tags used in the codebase.

Now that I have fetched the questions I have to implement the autocomplete feature. I did it using the jQuery typeahead function, similar to what was done before for the search feature.

The JavaScript code for typeahead goes as follows


$('#questions_searchform_input').typeahead({
items: 15,
minLength: 3,
source: function (query, process) {
return $.post('/questions_search/typeahead/' + query, {}, function (data) {
return process(data);
})
},
updater: function(item) {
var url;
if ($(item)[0] != undefined) url = $(item)[0].attributes['data-url'].value;
else url = '/questions_search/' + $('#questions_searchform_input').val();
window.location = url;
}
})

I have a input field with an id ‘questions_searchform_input’. The items option tells the maximum items that can be listed in the results. The minLength option specifies the minimum characters that will trigger the search. The source option gives the dataset of search results. Here we have used a ajax post request that submits the keyword to the /search/questions_typeahead/:id  url.

The actual work of fetching the results is done in the search controller. It has two actions questions and questions_typeahead. The questions action shows the results searched by the keyword and displays them on submit. If no match is found it redirects the user to the post form to post  new question. The questions_typeahead action generates the results for the autocomplete results.

You can find my work on the search functionality in the commit ceabfb0.

Lastly I wrote some functional tests for the search controller and also an integration test to test the entire search mechanism. You can find my full work in the plots2 Pull Request #555

This is just a simple search functionality. There is much more to it for developing a advanced search functionality. In fact there is a parallel Advanced Search Project going on in Publiclab lead by David Days. In course of time the search code will be modified to meet the advanced search needs.