The PublicLab Rich Editor

The twelfth week marker! The beginning of the end! GSoC 2016 will soon come to an end. I am almost done with my work on the Q & A system though some work is still left as per my timeline goals. The only major work that remains is integrating the PublicLab Rich Editor for use in the Q & A system.

The PublicLab Rich Editor is a separate project in PublicLab that my mentor Jeffrey Warren has been working on. It is an Editor that supports both markdown and WYSIWYG content. It is to be integrated in the publiclab.org website for posting content very soon. As a part of Q & A project I thought of working on it a bit and contribute as much as possible. It wasn’t initially included in the timeline when I initially made my proposal but later I modified my timeline to include it as it really seemed interesting.

The  PublicLab Rich Editor is a s a general purpose, modular JavaScript/Bootstrap UI library for rich text posting, which provides an author-friendly, minimal, mobile/desktop (fluid) interface for creating blog-like content, designed for publicLab.org. It uses grunt for packaging and compilation. It has a rich text editor based on the Woofmark library and an autocomplete  feature  supported with horsey library. It uses jasmine as its testing framework.

Since I am not familiar with Nodejs or npm it is a bit tough for me to understand its modular structure. I still didn’t make any contribution to the Rich Editor. But I will likely make some contribution as the program wraps up. Have to learn Nodejs for this. I also have to work on fixes that come up during the last week as things are going to be deployed in the live site.

Also my team mates who were working along with me in the same website but different project are doing some awesome work and there is big merge on the Search project that is coming up. So things are getting busy in plots2 in the upcoming week. Stay tuned as the GSoC 2016 comes to an end!

Testing with Jasmine

This marks the seventh of GSoC 2016 and a lot of work has been merged since I started working. Looking back at it gives real feeling of accomplishment.

An important part of writing good code is that it is well tested. So that when you start working on a new feature or a fix you ensure that it doesn’t break any functionality by running the tests. Having a good test suite ensures your app has a strong backbone and it also can be extended for further development.

We have been writing rails tests for quite sometime and now we have good test suite for plots2 . But we didn’t have any test framework for testing JavaScript functionalities. So my mentor Jeffrey Warren suggested that we have tests for JavaScript too. We chose the Jasmine framework as it is the most popular JavaScript framework used and it also comes out in different packages for in browser testing and npm package as well as a rubygem as well for headless command line testing powered by phantomjs. And the best thing is it is best suited for Rubyists as the testing pattern is similar to Rspec – the popular Ruby testing framework. Here I am going to brief on how to setup Jasmine using the jasmine-rails gem. Since we were mostly using jQuery for JavaScript functionalities we also used the jasmine-jquery-rails gem to enable use of html-fixtures and some custom matchers.

Setup jasmine-rails

Follow the installation instructions to install the jasmine-rails gem to use it in your test-suite. This should also install the phantomjs gem along with it. But if the package installation fails you follow this documentation given here. A  spec/javascripts  folder will be created where all test files will reside.

You can run tests headlessly using the command

rake spec:javascript

 

How to write Tests ?

Now the most important how to actually write tests? The best place to start is with the jasmine docs. It gives a comprehensive guide to how to write tests. Jasmine tests basically describe a test and set some expectations which are tested against some matcher. All tests are written in spec files which are basically js files that reside in the spec/javascripts folder. Let us take a very simple test. As you can see it is written in a js file jasmine_spec.js


describe("A suite", function() {
it("contains spec with an expectation", function() {
expect(true).toBe(true);
});
});

view raw

jasmine_spec.js

hosted with ❤ by GitHub

Here the describe() function start a test description with a string and a function that contains a block of code. The it block also starts with a string and a function that contains a code block that contains expectations. The describe block basically tells the bigger story. And the it block tells the smaller individual stories that need to be tested. The expect() functions defines the expectation that is to be made. It can contain any instance or message. Basically any variable or method that returns something should go in it. The expect()function goes with a matcher chained with it. Each matcher implements a boolean comparison between the actual value and the expected value. It is responsible for reporting to Jasmine if the expectation is true or false. Jasmine will then pass or fail the spec. Here toBe() is a matcher.

Faking Ajax Calls

Now there are Ajax calls in applications that require server side requests. But depending on server responses makes the test suite slow and also leads to test failures when offline. So to test Ajax we need to fake Ajax calls using mock-ajax. You can simply copy paste the code in this link and put it in a helpers/mock-ajax.js file that goes inside the spec/javascripts folder. To mock ajax calls across a test suite you need to use install() and uninstall() in beforeEach()and afterEach() blocks in the beginning of the describe block as explained in the documentation. This sets up jasmine-ajax across the test suite. You can have a look at the jasmine-ajax documentation to have an idea on how to fake ajax calls.

Basically what happens in the entire process is you make or trigger a Ajax request first. Then you need to “Spy” on the Ajax call and return a fake Ajax response.  After that you need to test that with the expectation. A Spy is nothing but a function that helps faking the response. It is similar like a stunt double of an actor who fakes for the actor for a shot in a movie! You can learn more about Spies here. I was using jQuery and was using the .ajax() function for ajax calls so i preferred this stackoverflow answer to fake the ajax response.

Now you have been thinking what’s the point of testing if we actually setup the fake response  ourselves and test the same. Its definitely going to pass! Well  it’s not going to pass unless you have your Ajax call working properly. The entire point is not testing the response but whether the Ajax call is actually triggered on executing the action. The response is tested in the server framework, here, the Rails test suite.

Using HTML fixtures

When you are testing JavaScript in a web application you must have it associated with some html element may its a button or a link. What you want is to test it against some event like a hover or a button press specifically those associated with an Ajax call. What you need to do is to create a html file with that specific element like the button or link similar to what you have written in the actual page and load it in the test suite. HTML fixtures go in a separate fixtures directory inside the spec/javascripts directory. Here is a sample fixture I wrote .


<ul class="btn-group">
<li rel="tooltip" title="Helpful? Like it and get updates!" class="btn btn-default btn-sm btn-like" node-id="1" id="like-button-1"><span id="like-star-1" class="fa fa-star"></span> <span id="like-count-1">1</span></li>
</ul>
<script>
jQuery(document).ready(function() {
$('#like-button-1').on('click', clickliked);
})
</script>

view raw

unlike.html

hosted with ❤ by GitHub

As you can see I also associated the event associated with the button so that I can trigger the button press in the test.

To load the fixtures include the fixtures path at the top of the describe block

jasmine.getFixtures().fixturesPath="../../spec/javascripts/fixtures";

Also if you are using jasmine-jquey you need to preload the fixtures by including this line

preloadFixtures('unlike.html')

Finally load the fixture at the beginning of the specific test in the it block

loadFixtures('unlike.html');

Now you can use the html elements in the test. Like you can trigger the button click using

$('#like-button-1').trigger('click');

That’s how you use fixtures in test.

Whenever you have a problem in testing try debugging using a console.log() message. That helps you in figuring where you are going wrong in the test.

Here is how a spec file would look. You can find my work on this in plots2 PR #618.

GSoC 2016: The Next Phase

I finally made it through the Mid-term evaluations of GSoC 2016! It makes me happy to have made it this far but it’s still not over yet. Moving on to the next phase of GSoC. This ones going to be important as it is longer than the first phase and also I have some pretty good features that I had planned out in my timeline. Here is what I have been working in the past week after the Mid-term evaluations. But before I go to that I want to show you what my mentors gave me as feedback during my Midterm evaluation.

midterm

Really some praising words! Its things like these that really motivates to go on with my work. And that’s an essence of open source too! If you are working with the right community, people really help you out and you get the word of appreciation.

The next feature I have been  working on is making an Accept button for answers. This makes it possible for the author the question to mark an answer as accepted. The “accepted” answer would be marked with a green label  when it would be accepted. The question author can also unaccept it by pressing the same button again. Here is the screenshot of how it would look.

accept_answer

The complete work can be found in the plots2 PR #598

 

Answering System for Publiclab – Expanded Q & A Project

Going over the third week in GSoC Coding period, a lot of Progress has been made over this week. A large chunk of Code has been merged and it makes me relieved that an important part of the Project is complete. This week I developed the Answering system for the Q & A Project which is the next most important part after the Questioning system.

Here is the summary of the tasks that I covered in the Answering system

  1. Create a new Answer model for handling Answers to each question
  2. Develop Post, edit and delete methods for Answers
  3. Design how Answers are listed for each Question in the Question page
  4. Create Like functionality for Answers
  5. Write functional and unit tests for the features implemented

These are the overview of the features implemented. Let me now go through the technical part of the implementations.

First is the Creation of the new Answer Model. As already mentioned in earlier posts all content in plots2 is handle by the DrupalNode model. Questions are a just type of node. Each question can have any answers so the Answer model is related to the DrupalNode model and also to a DrupalUser model that is the model which handles users in plots2 since an answer is written by an user. After setting up the relation, validations and various methods for the model had to be written for the model. Validations refer to the set of requirements that each record in the table must follow like the content of an answer can’t be blank. Rails defines a set of Validation methods for a model that handles this.

Next was creating a answers controller for handling the Post, edit and delete mechanisms for the answers. Answers are posted using Ajax requests in plots2. So is the delete action. Update action uses in-place edits for editing answers but the Update action is a normal http request. Working with Ajax in Rails is as simple as anything. Rails handles Ajax requests using non-obstrusive JavaScript. You just have to define a data-remote property in the links or forms for making Ajax requests and Rails will smartly handle each Ajax call. You can read more about it here.

Now coming to the Like functionality for questions. The Like button also involves Ajax requests. It is basically a button that Likes or unlikes an answer on button press. To implement the Like functionality a new AnswerSelection model had to be introduced. Eack like is related to the answer and the user who liked it. The answer_selection table has a liking field that takes a boolen value. It is set to true when an answer is liked and it is set to false if the answer is unliked. To deal with the Ajax calls I created a new answer_like controller which has a likes action that updates the liking field of the record.

Finally writing unit tests for the new models and functional tests for the new controller and implemented changes is a required part of good code.

On top of that I got to had a nice conversation with David Days and Ujitha of the Advanced search Team and moved the question Search functionality to a separate controller to avoid any conflicts and better collaborate with each other.

You can see the work on Answering functionality in plots2 Pull request #566.

Creating a Basic Search functionality

The second week of GSoC coding period is over and here are the works that I completed during this week.

This week was mostly spent on improving the design of the Questions page as proposed by my mentor and introducing a Basic question search functionality. I had to do some follow up fixes for my previous changes like adding comments for some temporary changes and one most important fix was to have a custom method defined for questions url so that it could be used at places required instead of writing out the full path every time.

Following the design proposed in plots2 issue #554 I modified the design for the questions page using some of the dashboard templates used in plots2.

You can now see the new questions page at https://publiclab.org/questions/ .

As proposed in the design I made a Basic Search functionality for Questions. Here I will walk through How I made the Basic Search functionality for questions.

There was already a search functionality that searched for notes, wikis, maps and comments from the  search box in the navbar. Autocomplete results are shown as you type in any keywords showing a icon and title in the autocomplete results suitably linked with the item. There is also an advanced search which you can see in the url  https://publiclab.org/search/ . There you can search selectively for any research note, wiki, map or comment by checking the checkboxes present there.

The question search is similar to the search except that it searches only questions. The actions for the questions search would go in the search controller. So the search query searches for any notes whose title matches the keyword and also has a question:foo tag. The search query for that is


@notes = DrupalNode.where(
'type = "note" AND node.status = 1 AND title LIKE ?',
"%" + params[:id] + "%"
)
.joins(:drupal_tag)
.where('term_data.name LIKE ?', 'question:%')
.order('node.nid DESC')
.page(params[:page])

view raw

search_query.rb

hosted with ❤ by GitHub

Here the :id is the search keyword typed. You can see my previous post for the definition of nodes and tags used in the codebase.

Now that I have fetched the questions I have to implement the autocomplete feature. I did it using the jQuery typeahead function, similar to what was done before for the search feature.

The JavaScript code for typeahead goes as follows


$('#questions_searchform_input').typeahead({
items: 15,
minLength: 3,
source: function (query, process) {
return $.post('/questions_search/typeahead/' + query, {}, function (data) {
return process(data);
})
},
updater: function(item) {
var url;
if ($(item)[0] != undefined) url = $(item)[0].attributes['data-url'].value;
else url = '/questions_search/' + $('#questions_searchform_input').val();
window.location = url;
}
})

I have a input field with an id ‘questions_searchform_input’. The items option tells the maximum items that can be listed in the results. The minLength option specifies the minimum characters that will trigger the search. The source option gives the dataset of search results. Here we have used a ajax post request that submits the keyword to the /search/questions_typeahead/:id  url.

The actual work of fetching the results is done in the search controller. It has two actions questions and questions_typeahead. The questions action shows the results searched by the keyword and displays them on submit. If no match is found it redirects the user to the post form to post  new question. The questions_typeahead action generates the results for the autocomplete results.

You can find my work on the search functionality in the commit ceabfb0.

Lastly I wrote some functional tests for the search controller and also an integration test to test the entire search mechanism. You can find my full work in the plots2 Pull Request #555

This is just a simple search functionality. There is much more to it for developing a advanced search functionality. In fact there is a parallel Advanced Search Project going on in Publiclab lead by David Days. In course of time the search code will be modified to meet the advanced search needs.

 

Parsing RSS feeds

There are many ways of Parsing RSS feeds data so that it can be used for showing useful information in a html format. Here I will show how we parse rss feed data using JavaScript that can be presented  in a webpage.

RSS is a type of document that uses standard web feed format for publishing frequently updated data like blog posts, headlines, audio or video. It is analogous to xml which just documents data primarily rather than rendering them. RSS feeds have a standard format of representation. You can find the specifications in this link. Here is a sample RSS 2.0 feed that documents map data for an author in mapknitter.org maintained by Publiclab.

To parse the rss feeds I used the jQuery.get() method which is function that handles Ajax requests. We use this method to load the rss feed using a get request


$.get('https://mapknitter.org/feeds/author/waren&#39;, function (feed) {
// processing code goes here
})

view raw

jQuery_get.js

hosted with ❤ by GitHub

As you can see in the RSS feed each item is surrounded by an <item> tag and the whole content is wrapped by a <channel> tag. The $.get() function on Ajax success returns the response in the feed variable. The feed variable thus has the content of the RSS feed.

Suppose we want to get the title and description of items in the feed. The processing code will then be


$.each($(feed).find('channel item'), function (i, item) {
title = $(item).find('title').html(),
description = $(item).find('description').html();
// Rendering code goes here
})

Similarly, we can get other elements of the item. Now we can use this data along with some JavaScript to render it on a webpage.

I had a table within a div with an id ‘maps’ to list all maps fetched from the rss feed data. So what I did was append some rows and cells to it to list them . The rendering code was similar to this


$('#maps table').append('<tr class="feed-item-' + i '"></tr>');
var itemEl = $('#maps table .feed-item-' + i);
itemEl.append('<tr class="title"></tr>');
itemEl.find('.title').html(title);
itemEl.append('<tr class="description"></tr>');
itemEl.find('.description').html(description);

So  you see it’s pretty simple Parsing RSS feeds using some JavaScript code. You can see my actual work in commit 4127568.