Creating Custom Rake Tasks

As I write this the tenth week in GSoC 2016 is over and it’s the end of the month now. I am a little late with my timeline mostly because I am doing work on the web interface for some pages and things need  to be changed over and over to fit the design choices. But in the mean time I got to learn something new. We had to run two separate rake tasks for running rails tests and JavaScript tests. So there was an issue to run all tasks with one rake task for ease. This is where I learnt about creating custom Rake tasks.

Rake is a Make-like program implemented in Ruby. It comes as a Rubygem and has to be included in your Gemfile during development. Custom Rake tasks start with a namespace followed by the task name separated by a colon. They reside in the lib/tasks directory of a rails app and they go in a file with a .rake extension.

Here is the railsguides documentation on How to create  Custom Rake tasks. Here is this good blog post by ANDREY KOLESHKO while I learnt about writing rake tasks. As you can see there you can either directly write the task in the namespace.rake (remeber to name the file same as the namespace ) file in the lib/tasksdirectory or you can use the rails generator to create the task file

$ rails g task my_namespace my_task

This will create a lib/tasks/my_namespace.rake file where you can write your task.

I needed to run the two tasks rake test and rake spec:javascript from this task. So basically I had to run rake tasks from within a rake task. Here is a stackoverflow answer that answers it perfectly! Know the difference between execute, invoke and reenable.

Here is a custom task I wrote. It can run using rake test:all

	# rake test:all
	namespace :test do
	desc "Run rails and jasmine tests"
	task :all do
	puts "Running Rails tests"
	Rake::Task["test"].execute
	puts "Running jasmine tests headlessly"
	Rake::Task["spec:javascript"].execute
	end
	end

view raw

test.rake

hosted with ❤ by GitHub

You can find this in the commit e90fe of plots2. I would be probably posting about the work on design changes in my next post.

Keeping Rails models DRY with concerns

Its over the ninth week in GSoC 2016 and here I am with another of my blog post. Today I will discuss about a pretty nice thing that I learnt while I was working with the notification system. It is related to writing DRY code by using of modules.

My mentor Jeff said that methods in some models were becoming repetitive and they were doing the same kind of task. So it would be better if we could keep the code DRY i.e. avoid redundancy in code. DRY stands for Don’t Repeat Yourself i.e. try to avoid repetitive code in a application. We can do this by  creating methods and modules. Ruby has an awesome concept of modules and mixins  for this purpose.

ActiveSupport Concerns follow the same principle  of Ruby modules and by extending them on Rails models we can include common methods that are shareable across various models.

We had two models named Node and  Answer models. A Node basically served as a container of posts like wikis, questions and research notes in publiclab.org. Answer as the name suggests are answers to questions. Both have some common methods named liked and liked_by. So we made a NodeShared concern and put those in them. Here is how the concern looks like

	module NodeShared
	extend ActiveSupport::Concern
	def likes
	self.cached_likes
	end
	def liked_by(uid)
	self.likers.collect(&:uid).include?(uid)
	end
	end

view raw

node_shared.rb

hosted with ❤ by GitHub

This file goes in a separate app/models/concerns/directory. This is where all concerns reside. To define it as a concern you need to extend the ActiveSupport::Concernmodule as I did above. After that after you have defined the concern you can use it in others models by including the concern like include NodeShared in this case. Here is how the node model would look

	class Node < ActiveRecord::Base
	include NodeShared # This includes the NodeShared Concern
	attr_accessible :title, :uid, :status, :type, :vid, :cached_likes, :comment, :path, :slug
	self.table_name = 'node'
	self.primary_key = 'nid'
	has_many :node_selections, :foreign_key => :nid
	has_many :answers, :foreign_key => :nid
	def latest
	self.revisions
	.where(status: 1)
	.first
	end
	def revisions
	self.drupal_node_revision
	.order("timestamp DESC")
	end
	end

view raw

node.rb

hosted with ❤ by GitHub

This is a small part of the actual model used. So now the methods in the concern become a part of the model and they can be called with an instance of Node model. Now suppose the Answer model also needs the similar methods. We can simply add this concern to the model as well instead of writing the methods over again.

DHH the creator of Ruby on Rails explained this in his blog post.

Coming back to my progress in Summer of Code I have started working on UI changes and modified views that will include some good features. I will discuss them in my future posts. You can look at my PR #628 though it is Work in progress now. As the mid of July is over its probably going to be more work in the next couple of weeks as things are nearing the end.

Email notifications using Rails ActionMailer

The eighth week of GSoC 2016 coding period is over and I got to work on something new in Rails. This time I was making the email notification system for the Q & A system so that when a user posts a question or answer related people get notified via emails.

Rails has has email delivery framework called ActionMailer. Though I was familiar with Rails, I haven’t worked on ActionMailer before this. So the first thing to do was checking the ActionMailer documentation. I strongly recommend following this  documentation to  understand the email delivery system. Rails guides have good documentation for each version of Rails and the features well explained. It’s a good place to start with.

I had to create a new AnswerMailer and modify the existing CommentMailer and SubscriptionMailer to send out email notifications to users.

A Mailer basically contains some methods that defines the recipient and the subject of the mail to be sent along with the delivery method. It inherits from ApplicationMailer which basically contains the default sender mail address. The mailer view template goes under the views directory which is the standard template directory for Rails. The view template is named according to the mailer method. So for AnswerMailer if there is a method named notify_question_author the template would go with the name notify_question_author.html.erb in the views/answer_mailer directory. Whatever content you want to send in the email goes in this email. So you see the naming convention and functioning quite is similar to Rails controller though not the same.

To send a mail you need to call the mailer method from the controller as described here. Now you need to send mail on specific conditions. Like when you create a post, for example, you would want the subscribers to get specific content while when you post an answer to a question you would like to notify the question followers as well who asked the question. So it is good to create a class or an instance method in related model to do so. Like I defined a instance method answer_notify() for Answer model that notifies the question author as well  as users who liked the question. This contained  two mailer methods AnswerMailer.notify_question_author() and AnswerMailer.notify_answer_likers() which sent emails on meeting some specific conditions. So this way you can frame a custom method that sends the mail and suits the conditions as well keep the code in the controller simple an let the Model contain the method description.

You can find my entire work on this in plots2 PR #612.

GSoC 2016: The Next Phase

I finally made it through the Mid-term evaluations of GSoC 2016! It makes me happy to have made it this far but it’s still not over yet. Moving on to the next phase of GSoC. This ones going to be important as it is longer than the first phase and also I have some pretty good features that I had planned out in my timeline. Here is what I have been working in the past week after the Mid-term evaluations. But before I go to that I want to show you what my mentors gave me as feedback during my Midterm evaluation.

Really some praising words! Its things like these that really motivates to go on with my work. And that’s an essence of open source too! If you are working with the right community, people really help you out and you get the word of appreciation.

The next feature I have been  working on is making an Accept button for answers. This makes it possible for the author the question to mark an answer as accepted. The “accepted” answer would be marked with a green label  when it would be accepted. The question author can also unaccept it by pressing the same button again. Here is the screenshot of how it would look.

The complete work can be found in the plots2 PR #598

 

Commenting System for Publiclab – Expanded Q & A Project

The fourth week of GSoC Coding Period is also over and the Mid-term evaluations are just about to come. So coming Mid-way of the GSoC journey, here are the final tasks that I completed before the Pre-midterm period.

1. Modify existing comment system to handle answer comments and question comments
2. Create new views and styles for comments in the Q & A pages
3. Create expandable text boxes for comments and a View more functionality for comments

These are the brief overview of the features implemented. Let me now come to the technical details.

The current commenting system for Research notes uses a DrupalComment model for handling the comments in the database. So I used the same model for answer comments too. For answer comments I just had to introduce a aid column in the comments table and relate comments to answers. But I had to take care of a thing that answer comments weren’t related to the question otherwise all the answer comments would also show up in the question comments section in the page. So what I did was store a 0 in the nid field of answer comments to unrelate any answer comments to questions.

I also used the same comment controller for hadling requests for answer comments with some modifications. Took me to create new js partials for answer comments as comments were created and deleted witj Ajax requests.

The next work that I completed was introduce a new comments view for the Q & A pages. One of the new feature was creating a View more button for questions so that more comments were shown on pressing the View more button. Comments are listed in the descending order of their creation. So recent comments are listed at the top and older comments are shown as you Click View more.

Another cool feature that I implemented was expanding text boxes for comments. Did you ever notice how text boxes of facebook comments expand as you type in more lines without giving a annoying scrollbar. It’s the same feature that I implemented. I followed this article for that. I just used jQuery instead of raw JavaScript. It’s a really good feature when you want to save space as well give a good User Experience.

That’s all before the Mid-term evaluations. You can find this work in plots2 Pull request #589

Answering System for Publiclab – Expanded Q & A Project

Going over the third week in GSoC Coding period, a lot of Progress has been made over this week. A large chunk of Code has been merged and it makes me relieved that an important part of the Project is complete. This week I developed the Answering system for the Q & A Project which is the next most important part after the Questioning system.

Here is the summary of the tasks that I covered in the Answering system

1. Create a new Answer model for handling Answers to each question
2. Develop Post, edit and delete methods for Answers
3. Design how Answers are listed for each Question in the Question page
4. Create Like functionality for Answers
5. Write functional and unit tests for the features implemented

These are the overview of the features implemented. Let me now go through the technical part of the implementations.

First is the Creation of the new Answer Model. As already mentioned in earlier posts all content in plots2 is handle by the DrupalNode model. Questions are a just type of node. Each question can have any answers so the Answer model is related to the DrupalNode model and also to a DrupalUser model that is the model which handles users in plots2 since an answer is written by an user. After setting up the relation, validations and various methods for the model had to be written for the model. Validations refer to the set of requirements that each record in the table must follow like the content of an answer can’t be blank. Rails defines a set of Validation methods for a model that handles this.

Next was creating a answers controller for handling the Post, edit and delete mechanisms for the answers. Answers are posted using Ajax requests in plots2. So is the delete action. Update action uses in-place edits for editing answers but the Update action is a normal http request. Working with Ajax in Rails is as simple as anything. Rails handles Ajax requests using non-obstrusive JavaScript. You just have to define a data-remote property in the links or forms for making Ajax requests and Rails will smartly handle each Ajax call. You can read more about it here.

Now coming to the Like functionality for questions. The Like button also involves Ajax requests. It is basically a button that Likes or unlikes an answer on button press. To implement the Like functionality a new AnswerSelection model had to be introduced. Eack like is related to the answer and the user who liked it. The answer_selection table has a liking field that takes a boolen value. It is set to true when an answer is liked and it is set to false if the answer is unliked. To deal with the Ajax calls I created a new answer_like controller which has a likes action that updates the liking field of the record.

Finally writing unit tests for the new models and functional tests for the new controller and implemented changes is a required part of good code.

On top of that I got to had a nice conversation with David Days and Ujitha of the Advanced search Team and moved the question Search functionality to a separate controller to avoid any conflicts and better collaborate with each other.

You can see the work on Answering functionality in plots2 Pull request #566.

Creating a Basic Search functionality

The second week of GSoC coding period is over and here are the works that I completed during this week.

This week was mostly spent on improving the design of the Questions page as proposed by my mentor and introducing a Basic question search functionality. I had to do some follow up fixes for my previous changes like adding comments for some temporary changes and one most important fix was to have a custom method defined for questions url so that it could be used at places required instead of writing out the full path every time.

Following the design proposed in plots2 issue #554 I modified the design for the questions page using some of the dashboard templates used in plots2.

You can now see the new questions page at https://publiclab.org/questions/ .

As proposed in the design I made a Basic Search functionality for Questions. Here I will walk through How I made the Basic Search functionality for questions.

There was already a search functionality that searched for notes, wikis, maps and comments from the  search box in the navbar. Autocomplete results are shown as you type in any keywords showing a icon and title in the autocomplete results suitably linked with the item. There is also an advanced search which you can see in the url  https://publiclab.org/search/ . There you can search selectively for any research note, wiki, map or comment by checking the checkboxes present there.

The question search is similar to the search except that it searches only questions. The actions for the questions search would go in the search controller. So the search query searches for any notes whose title matches the keyword and also has a question:foo tag. The search query for that is

	@notes = DrupalNode.where(
	'type = "note" AND node.status = 1 AND title LIKE ?',
	"%" + params[:id] + "%"
	)
	.joins(:drupal_tag)
	.where('term_data.name LIKE ?', 'question:%')
	.order('node.nid DESC')
	.page(params[:page])

view raw

search_query.rb

hosted with ❤ by GitHub

Here the :id is the search keyword typed. You can see my previous post for the definition of nodes and tags used in the codebase.

Now that I have fetched the questions I have to implement the autocomplete feature. I did it using the jQuery typeahead function, similar to what was done before for the search feature.

The JavaScript code for typeahead goes as follows

	$('#questions_searchform_input').typeahead({
	items: 15,
	minLength: 3,
	source: function (query, process) {
	return $.post('/questions_search/typeahead/' + query, {}, function (data) {
	return process(data);
	})
	},
	updater: function(item) {
	var url;
	if ($(item)[0] != undefined) url = $(item)[0].attributes['data-url'].value;
	else url = '/questions_search/' + $('#questions_searchform_input').val();
	window.location = url;
	}
	})

view raw

typeahead_example.js

hosted with ❤ by GitHub

I have a input field with an id ‘questions_searchform_input’. The items option tells the maximum items that can be listed in the results. The minLength option specifies the minimum characters that will trigger the search. The source option gives the dataset of search results. Here we have used a ajax post request that submits the keyword to the /search/questions_typeahead/:id  url.

The actual work of fetching the results is done in the search controller. It has two actions questions and questions_typeahead. The questions action shows the results searched by the keyword and displays them on submit. If no match is found it redirects the user to the post form to post  new question. The questions_typeahead action generates the results for the autocomplete results.

You can find my work on the search functionality in the commit ceabfb0.

Lastly I wrote some functional tests for the search controller and also an integration test to test the entire search mechanism. You can find my full work in the plots2 Pull Request #555

This is just a simple search functionality. There is much more to it for developing a advanced search functionality. In fact there is a parallel Advanced Search Project going on in Publiclab lead by David Days. In course of time the search code will be modified to meet the advanced search needs.

 

Questioning System for Publiclab – Expanded Q & A Project

The GSoC Coding Period has already started and its been a week since then. This post contains the work that I have done during the first week of Coding period.

Here are the list of tasks that I worked on during the first week

1. Create a questions page that lists all the questions asked in the community.
2. Create sections in questions page that shows Recently viewed, Most viewed and Most liked questions. They are shown in Recent, Popular and Liked tab respectively.
3. Create a questions show page that displays the question along with its description.
4. Placing buttons for editing, deleting and liking the question in the show page.
5. Modify post mechanism for questions.
6. Writing unit tests for the implemented changes.

Coming to the technical point of the implementations let me just walk through the Basic structure of the resources used for posting Research Notes in Publiclab.

All Research notes and wiki pages in plots2 codebase  are stored as nodes and accessed by the drupal_node model (Don’t think this has something to do with drupal. They probably migrated from drupal to Rails hence the name). In the current codebase a node has basically three ‘types‘. It is either note, wiki or maps.

plots2 has a tagging system where tag names are stored in a term_data table accessed by the drupal_tag model. There are some tags called power_tags that have a name like something:foo.

Questions are notes that have a question power_tag i.e. the tagname is question:foo. So for listing questions in the question page all I had to do was to list the notes with such tags. I had to create a questions controller for this purpose. The Recent tab listed questions in the descending order of post creation date. The Popular tab listed questions in descending order of the number of views. The Likes tab listed questions in the descending order of the number of likes.

Since there was already an posting mechanism and questions were actually a special kind of notes I had to just reuse the code and eliminate any redundancy. Any piece of software you write must always take care of this. Rails already supports this by writing DRY(Don’t Repeat Yourself) Code. So for post, edit and delete actions I had to modify the notes controller it so that it could work well for questions. Since the work is in the development phase I also had to take care that users don’t accidentally land up on these pages while they try to post a question or edit it that have a question:foo tag. This was taken care off by appending a :redirect parameter in the url of post and edit paths for the new feature. The post request would land up on the questions show page only if there is :redirect parameter in the url set to question.

Another bug that I solved in the way was that previously pages that required logged in access didn’t redirect users to the specific page rather they were redirected to the user dashboard page. I solved it to make it running. This could be used for the Ask a question button that required log in. Once the users are logged in they were taken to the post form.

Finally I had to write thorough test code for all the features implemented. Any codebase that needs to be used in long run must have thorough testing. My mentor Jeff  also supported this fact. Though I had experience of writing tests in Rspec, a Ruby testing framework the plots2 repository used Rails test::unit for testing purposes. So I started learning how to write tests in test::unit. It is really good to see the green dots for passing tests that you write! Once you get a hang of it it’s really fun writing test code! The Rails testing guide has some really good documentation on writing tests.

You can see my full work on this in plots2 Pull Request #550.

Creating Hashtags

The GSoC Community Bonding period has already started and I am really excited to be a part of it. This week I got to work on a issue for Creating Hashtagging System. It’s really simple. All you have to do is to find words that are preceded by a # and replace them with suitable links.

For matching keywords preceded by a Hashtag I used a Regular expression, keywords that would match the regular expression was replaced with suitable linking markup. I am working on Rails framework. So I will show here how to write Ruby code for implementing Hashtags. If you are new to Rails you can perhaps see how to setup a development environment in my previous post.

I am defining the Regular expression in a constant defined in the config/initializers/constants.rb file. It is a general convention to define constants that are used by the application in files in config/initializers directory. The regular expression that I used was

HASHTAG = /(\s)\#([\w-]+)/

What the Regular expression means is that, it should match words that has a space before a ‘#’ followed by any word character (i.e. letter,number or underscore) or dash. The constants were defined under a module named Callouts. Ruby modules are like wrappers around some class or methods. So we use the module to access these constants. The next thing is to define the markup that was to be replaced in place of the keywords. So that is also defined in the constants.rb file under the Callouts module. In the present case we had two formats for linking one for use in markdown and one for linking as html markup. So the markup texts goes as follows

HASHLINKMD = '\1[#\2](/tag/\2)'
HASHLINKHTML = '\1<a href="/tag/\2">#\2</a>'

Matches for Hashtags are generated in two groups. as you can see above the HASHTAG has groups covered in round brackets. The first group matches a space and the second group has the keyword. The markup texts uses these groups as ‘\1′ and ‘\2′.

Now we have defined the matcher and replacement markup, we actually need something to make it happen. Here comes the role of gsub. gsub is a method of String class that replaces the all the matches found within the string with the second argument given. The first argument is generally a Regular expression e.g.

"hello".gsub(/[aeiou]/, '*')                  #=> "h*ll*"

So to replace the hashtag with the appropriate markup I used

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKHTML))

where body is a String class object.

Similarly to replace the hashtags with the markdown markup the code will be

body = body.gsub(Callouts.const_get(:HASHTAG), Callouts.const_get(:HASHLINKMD))

I have used the method const_get to fetch the regular expression. const_get finds a constant with given name in the specific module or Class.

Hurray! We have successfully made a Hashtagging system. Now you can show the desired content in the page linked to the Hashtags.

Well there was much more to it. There were many edge cases that I had to test for. My mentor Jeff helped me point it out. You can see them here. as we were dealing wit markdown I had to take care that Headings weren’t linked as Hashtags. The key was choosing a correct Regular expression. You can test Regular expressions in Rubular as I did here. You can see my full work in commits 11e0dd5 and 6bd065a.