Serving multiple websites from sub routes of a domain using Nginx

Few days earlier I was working on migrating the PyConf Hyderabad website from GitHub Pages to a DigitalOcean server in order to achieve archive support for the previous year and current website. The websites are static websites generated by Jekyll.

Objective

The objective was to serve various years of the conference websites in year wise sub routes. The domain was pyconf.hydpy.org . So we wanted something like pyconf.hydpy.org/2017/ , pyconf.hydpy.org/2019/ . We had two separate repositories for both of the websites which we wanted to serve in the stated manner.

Solution

Nginx directives have an excellent way of accomplishing this by leveraging the power of regular expressions. Nginx is being widely used for most of the production level web servers. I was also using Nginx as the server in the DigitalOcean droplet.

Since the websites were in two different repos all I had to do is – whenever a request comes with a specific subroute I would point it to the specific directory path. This was done in the Nginx location directive. I had to change the Nginx config with the following similar code

location = / {
    rewrite "^.*$" /2019/ redirect;
}

location ~ ^/2017(.*)$ {
    alias /home/pyconf/hydpyconf2017/_site/$1;
}

location ~ ^/2019(.*)$ {
    alias /home/pyconf/hydpyconf2019/_site/$1;
}

Let us understand this step wise. There are two parts of this configuration

Step 1

Redirect home url to /2019/ domain . This is to redirect the root domain to the sub route of the present year website. The = modifier after the location directive matches the exact route which uses the rewrite directive to redirect it to the specified replacement expression.

Step 2

The ~ modifier is used for case sensitive regular expression matching. Here we specify that if we match a route starting with /2017 search that path in the directory specified by the alias directive. The $1 refers to the first matched group in the regex. We can use the groups in regex to form the replacement strings that we want to point to.

Thus you can see with the help of regular expressions we are able to serve two different static websites using Nginx directives.

Science Hack Day India 2019 Experience

© Sayan Chowdhury

This was my first year attending Science Hack Day India. I came to know about Science Hack Day after I met Praveen at FOSSASIA Summit 2017 at Singapore. He started the Science Hack Day (aka SHD) event in Belgaum, the city where he lives. This year was the 4th Edition of SHD India. The idea of Science Hack Day is to make anything in 24 hrs using Science. But it’s actually more than than. You will get to know how it turns out at the end of this post.

Day 1

I booked my tickets for SHD India 2 weeks before the conference. I reached the venue on Saturday, 19th Oct. My bus was 2 hrs late. The venue was in Sankalp Bhumi Resort in Parvati Nagar, Belgaum where SHD India is held every year. Once you reach the team picks you up from the bus stop to their venue, but since I was late Praveen guided the Auto driver to take me stratight to the venue. After I reached I met few students who come each year to the event. I met Sayan at the Breakfast area which they call ‘Dhaba’ . After Breakfast I went to my room to freshen up and get ready for the event. The venue of SHD India is really picturesque. It’s calm and quite place full of greenery around with some awesome spots for Hacking ! Here is a image I took of the cottage house I was living.

I went to the stage where the event was about to start. I met Jithin and Saptak there. They were going to work on a robotic arm powered by KuttyPy. Jithin is the maintainer of KuttyPy and he showed how it is used to send signals to a output port of a ATmega32 microcontroller by simple clicks of a button using the KuttyPy gui.

The event kicked off by introducing Science Hack Day. After that Mentors introduced their hacks to the audience and welcomed everyone interested to join their hacks. I introduced about the Papercraft Spectrometer by PublicLab which I was going to make for the event. There were many young hackers among us who were excited to Hack among us and also introduced their Hacks. After that I went to the Hacking area at the ‘Cafeteria’ . I met Hitesh who was hosting a CTF . It was the first time a CTF event was being organised at SHD. Me, Sayan and Saptak started solving the CTF puzzles. The problems were intriguing and we kept solving them one by one. Other hackers were working in some interesting stuff. I met Akshai M who was working on a LoRaWAN Transmitter. We breaked for lunch around 2 pm. The weather was cloudy and it was raining for most part of the day. Post lunch there was a Python 101 Workshop for college students hosted by Sayan. I helped people around to Install Python in their systems and writing their first Python Program. After that I continued solving CTF questions while others continued their hacks. Here is a photo of everyone busy Hacking 🙂

In the evening there were Lightning Talks. There were all sorts of lightning talks starting from Hardware Systems, Security and Privacy over Web to Gardening plants. I gave a lightning talk on my HydPy community experiences on how regional communities play an important role on engaging more people towards a shared objective and idea. It was the end of the day. We had Dinner after that where we had more interesting discussions. I returned to my room since it was raining quite a lot, where everyone (Me, Praveen, Sayan, Saptak, Akshai, Hari, Jithin) gathered beside the balcony of the cottage house discussing their experiences in FOSS and Conferences. It was bed time soon, end of an exciting first day.

Day 2

The next morning we had a video conference with the SHD San Francisco Team at 9.30 am. SHD was also happening in the same days at San Fransico this year. I woke up and went for breakfast with Sayan. The item served for breakfast was unique for me – Mishar and bread, a typical Maharashtrian breakfast item. After having breakfast everyone gathered near the Cafeteria for the video call. Ariel who is the global Director of SHD was hosting the call from San Francisco. Everyone was excited to share their hacks they were working on. It was really amazing to see people from 2 different timezones sharing there awesome hacks and their excellent level of enthusiasm. After the call ended everyone started working on their hacks again. They were to be presented soon to the audience so there was little time left. I went to take some materials for my Papercraft Spectrometer. There was a team which made the Da Vinci Bridge using wood. People were walking over it turn by turn. I also had a walk over the bridge. Hari was taking a Unicycle riding session for people.

Meanwhile there was a microcontroller Workshop using KuttyPy going on in parallel which was hosted by Jithin. My hack was almost complete and I was seeing hacks of other people. A team made an awesome Ruben’s Tube where we could see flames dancing with music. Shortly after everyone gathered for the SHD India group photo in the lawn. There was a Treasure Hunt event after that. Reminded me about my college days. Hackers were giving final touches to their hacks which were to be presented after lunch.

After lunch everyone gathered near the stage and get their hacks ready for the audience. I got Table 1 where I was talking to people Regarding Spectrometry and my Spectrometer hack. It was nice to talk to people from all age groups. There were some really good hacks around. Praveen was demonstrating Hacks using Liquid Nitrogen, breaking flowers after freezing them with Liquid Nitrogen. It was really spectacular to see science Lab experiments done in fun but careful way.

After the demonstrations were over it was time for Awards for the Best Hacks. The Da Vinci Bridge and Ruben’s Tube were among the Award winning Hacks. The Best Hack of the event was given to a Team who made colourful clay plasters . There were more Awards coming in. Me, Sayan and Saptak were jointly awarded the CTF Award for scoring 950 points individually. I got a medal for that 🙂 Finally it was time for the closing address, thanking all organisers and Sponsors and See everyone at next SHD. Fun events continued after that. There was a Domino sequence made by some students. It was really nice. Then there was Bubble blowing by participants. Enthusiasm was still high among the participants and Hackers.

Also it was time for me to say goodbye to friends at SHD. I had my bus at 9.15 pm. It was raining quite a lot. Before I left for my room for packing my things I spoke to Akshay Deshpande who was a mountaineer and doing Treks at Himalayan ranges. He was sharing anecdotes about his mountaineering experiences. He took a session on Mountain Raffling in SHD. Then there was music and lights and people dancing together. I had my dinner and after saying goodbye to everyone at SHD it was the end of event for me. Praveen arranged by drop at the Bus Stop. He has been a Super Host !

I really loved my first Science Hack Day. It was a weekend full of Hacks, Fun and exciting discussions beside the tranquil venue at Sankalp Bhoomi. Making friends is the best part of the event which I look out whenever I go to Conferences and meetups. Eagerly waiting for the SHD India event !

PyCon India 2019 Experience

What an experience it was! This was my second time attending PyCon India and third time attending a Python Conference. I have been a part of the organising team of PyConf Hyderabad 2017 and PyCon India 2018 (Hyderabad). Each time it has been a fulfilling experience and numerous takeaways and new bonds of friendship made. I haven’t blogged about my experiences in past conferences. So this will be my first blog post regarding my Conference experience.

The last 2 times I had attended Conferences close to where I was presently living. This time PyCon India was held in its 11th Edition at Chennai. Being organiser in past Conferences I knew about the wonderful experiences that I was involved, so I decided to be part of the Volunteering team for this year’s PyCon. I was eagerly waiting for the event from when it was announced. PyCon had become a mandatory event for me now, where I get to meet old friends and make new ones.

Day 0

I reached Chennai on 11th Oct night, before the Conference to help out with volunteer activities. I went straight to the Conference venue at Chennai Trade Centre where many volunteers had gathered for helping out with Pre Conference activities and goodie bag packing. This is like a fun event where we introduce ourselves and assign tasks for the upcoming days. I met HydPy folks – Ram, Murthy, Gokul; Dgplug friends – Rayan, Devesh, Chandan, Bhavin and many more friends whom I met at last PyCon. Chandan was co-ordinating the Volunteer team. I also met Vijay, Naren and folks from Chennaipy who were the organisers this year. After Dinner we started packing the goodies bags and sorting Attendee ID cards and getting prepared for the big day. It was almost 2 am when we left the venue to our hotel. Unfortunately my Oyo booking was cancelled, thanks to Murthy who shared his accommodation for the night.

Day 1

It was the first day of the Conference. I reached the venue around 8 am, there were already people coming in for registration. So I sat at the Registration Desk to help with Registration. I met with Kuntal (@hellozee) from Dgplug and Rayan who helped with the Registrations. Within no time there was a big queue of people. After sometime when the crowd lessened I went for Breakfast where I met Chirag, who came for the day. We discussed regarding the Upcoming PyConf Hyderabad 2019 event. After that I met all folks from HydPy who had come up for the conference. Here’s our group photo

After that I went to attend Pradyun‘s talk on Python Packaging . It was a really insightful talk explaining the pip ecosystem. Soon after the talk I started going around Sponsor booths to know more about their business and also to solve the puzzles they were gibing out to get some goodies 🙂 This year PyCon India had a long list of Sponsors – 41 of them ! Kudos to PyCon India team!

Post lunch there was Poster Presentation Scheduled where I was going to talk about HydPy Community. Also as I was a volunteer helped in setting up for Poster sessions as well. Folks from HydPy were also there, we also talked about our upcoming conference – PyConf Hyderabad on Dec 7-8. The day got over with ASL Devi‘s Keynote regarding Bridging gender gap in tech. It was an eye opening talk on different gender stereotypes and the unconscious biases that we have in our mind and how we can overcome this together. After that we had Volunteer’s meet to retrospect and discuss feedback regarding the day. The day ended with the Volunteer and Speakers dinner.

Day 2

My Day started again in the registration desk where I sat down to help with Lightning Talk registration and giving out ID cards to the attendees who missed taking it last day. Shortly after I went to the First Keynote of the day by Ines Montani . The Talk was titled Let Them Write Code . The Talk covered some best practices in development and coding, her Lessons from Open Source and much more. We had the Dgplug Staircase Meeting today which is a customary meeting that we have in PyCon India each other where we meet and greet people of Dgplug Community whom we have always seen in #dgplug IRC channel. It’s the time when we actually meet and talk to these people we see online. Kushal‘s absence was felt this time. Sayan was hosting this meeting and he discussed some important concerns regarding the community and the need for people interacting more in the IRC channel. After that he distributed Dgplug T-shirts which we had ordered before the Conference. We had the mandatory Dgplug group photo after that 🙂

I was helping out at the Helpdesk mostly rest of the day and interacting with people whom I met. There was the PyLadies booth just beside the Helpdesk where I talked to women from PyLadies Chennai community. They had a PyLadies Lunch last day and a Speed Mentoring Session to invite more Women in Tech. Finally we were up to the Final Keynote of the day by David Beazley . The Talk was titled – A Talk Near the Future of Python. After he started his Live Coding session in the Keynote the audience was simply going ga ga over his performance. He Live coded a Stack Machine which he later turnd into a Web Assembly compiler which played a Rocket Game at the end. He showed Python has a bright future by demonstrating the power of Python + Webassembly. This was The best Talk I have ever attended till date. He already published the screencast. Do have a look and amaze yourself. The thing that most impressed me most was his fearless stunt in front of 1200+ attendees and calmly handling the intermittent crashes and debugging them.

Finally PyCon India ended with Vijay‘s closing Address who was the Chair for this year’s PyCon. It was the end of the Conference after which we there was Workshop + Devsprints for next 2 days. At the end we had group photo with all the attendees.

PyCon India has been successful for the large pool of people who volunteer at the event and come from all parts of the country for the love for the Community. It has been a pleasure to work and interact with these amazing people. I met Noah this year who came all the way from Taiwan and volunteered from Day 0 for the event. It really amazes me to see the enthusiasm of the people for the Community.

Day 3

Oct 14-15 were Workshop + Devsprint Days. The venue was IIT Madras Research Park. I was also leaving tonight so I packed my bags and went to the venue in the morning. I started helping with the registrations for Workshops and Devsprints and guiding people to the respective rooms. Then I went to the Devsprint room to work on the Python Packaging Sprint which was hosted by Pradyun. I was eagerly waiting for that. I started with the setup and then working on a issue. After Lunch I found David Beazley in the venue so I quickly joined for a quick conversation with him. I asked the secret behind his Live coding stunt last day. He said he did roughly 15 iterations of the run, even spent 45 mins to debug an issue in one iteration. But yeah the fluency came over his 35 years of experience ! Also as he frequently took classes with group of students where he used to live code during his classes. It was a real pleasure to talk to him. After Lunch I continued to work on another issue from pip till the end of the day.

Finally it was time to say goodbye to friends and the awesome 3 days taking back lots of memories and experiences. PyCon India has become an event close to my heart now and it’s an event I don’t want to miss. See everyone again in next year 🙂

Summary

PyCon India has let me meet and communicate with people from all parts of world and domain. There is something to learn in each of these events. It’s An event by the Community. For the Community. My takeway this year was – Work, Interact and Share your experiences. You can always be a good at coding and be an excellent programmer, but unless you interact with more people and share your learning you cannot be a good person. Also Volunteer and give back to the Community. Volunteering requires time and effort, but its a gesture to give back to the Community and meeting people who are doing amazing work for the Community. The entire Open Source world largely revolves around people volunteering their time for the good of the Community. So even a small effort towards that can make a big change.

That’s all about my PyCon India 2019 experience . Do leave your comments ! And if this excites you do come next year to PyCon India 🙂

First BangPypers Meetup

BangPypers , the Bangalore Python User Group is one of the largest and oldest Python User Group in India. It has been running successfully for more than 10+ years. Most of the known faces in Indian Python Community have been a part of this meetup group at sometime or other.

After I moved to Bangalore I have been on the look out to attend the meetups but I missed it in the first 2 months since some work used to come by on the meetup days. Last Saturday, 21st September, I finally made time to attend my first BangPypers meetup.

The meetup was schedule at 10.30 am at Visa Technology Centre, Bagmane Tech Park. I decided to go by Cycling since I was also looking for a long ride in my Cycle for a while. But alas it took longer than expected owing to a wrong turn in my way.

The theme of this week was Design Thinking and it had 3 Talks scheduled. I reached around 11.15 am almost missing the first Talk which was a Design Patterns 101 Talk. The Speaker was almost in his closing notes. Nevertheless I found a place to seat in the almost full room.

I took out my phone to take notes and waited for the next talk which was on Design Pattern for distributed Architecture . The Talk covered various points on best practices related to distributed systems and tools and techniques to achieve them. Discussion revolved around Horizontal Scalability, Responsiveness, Security, Failure Handling, Centralised Logging, Metrics, Request Tracing, Health Checks, Configuration and Discovery. Since I never dealt with production level Python applications or Distributed systems the terms were quite new to me, few of them I encountered in past but didn’t quite fiddle in depth. I made a note of the discussion for future reference.

The last talk of the day was Organize your bookshelf using Micropython by Vinay Keerthi who was one of the Hosts at Visa, which is also a PyCon India 2019 accepted Talk. He made a LED book shelf organiser which would tell him the position of a book in the shelf by lighting LED array. He used Micropython on a NodeMcu in which he built a simple Flask app which would fetch data from a Postgresql database and store it in a queue which would send signals to the LED. I really liked the idea and it was a really hacky way to find a book. After his presentation I talked to him to get some pointers on starting my first Micropython project which I plan to do in the upcoming days.

After the session I talked to the organisers Anirudha and Abhiram with whom I shared my experiences at Hyderabad Python Community and also how they conduct BangPypers meetups. Visa had arranged lunch for the attendees so the discussion continued at the Lunch table where I got to talk with other attendees and interacted about the work they were doing. Bangalore has a vibrant developer community and there are many more communities like PyData, PyLadies which conduct regular meetups. PyLadies Bangalore are going to have a meetup on coming Saturday, September 28. Here is the announcment about the same.

At the end Vinay shared some more stories of his Hacks and experiences with tinkering with Micropython. And yeah here are some photos of the meetup I took

The thing that made the meetup nice was the great Venue and the Topics of the Talks. With the boom of ML/AI in Python Land having Talks that are unlike those, but most fundamental topics is good. Hope to attend future meetups as well!

Lastly I rode back home completing my first 20km streak in Cycle 😛

I also tweeted about the meetup after I returned home that day.

Takeways from the Meetup

  • Design Patterns in Python – Spend more time learning these.
  • Got to know about best practices in distributed system – Keep notes for futute projects.
  • Micropython getting started – Order some NodeMcu and get the hands dirty. TO DO: Read blogs regarding starter projects.

DevConf India 2019 experience

One month ago I attended DevConf India 2019. It was held from August 2-3 at Christ University, Bangalore. It was quite a while since I attended a conference, the last one being PyCon India 2018 . Due to my laziness in writing I have altogether missed writing my Conference experiences till now. This will be in fact my first Conference blog post. Now on I will make sure to write about each of the conferences I go.

I came to know about DevConf India last year when it was first held, from people in my developer circle. DevConf India is organised by Red Hat and has similar events in US and Czech Republic. Since I was in Bangalore this year I made sure I attend it after I saw the dates in Twitter. I registered as soon as I came to know about it.

Day 1

I mostly planned on meeting up with people and attend a few talks. I started at 8.30 am from my place and unfortunately missed the Keynote owing to a bad experience with a Bike Taxi service. I reached the Venue around 10 am and collected my Attendee badge and T-shirt. Then I headed towards the Keynote Session Hall where I met Naren from Chennaipy. I earlier met him at PyCon India 2018. It was nice catching up with him.

After having breakfast at the venue I headed to the Booth area where I met Chandan . I started visiting the booths asking questions about various projects like Fedora, Debian, CentOS. Shortly after I met up with some more familiar faces from DgplugSayan, Rayan , all of them I met during PyCon India last year. I expected a Dgplug staircase meeting at DevConf but unlike last year there were less attendees this time. After that we went for lunch at the Cafeteria where I met Ramki , Praveen and pjp . Few days earlier I was reading pjp’s tutorial regarding gcc and gdb from dpglug irc logs . It was nice to catch up with him in person. I was discussing with Sayan and Praveen about the initial days of dgplug at my college at NIT Durgapur, attending their first talk in 2013 when I had just joined my college.

After Lunch I decided to attend few talks. I attended a talk regarding Evolution of Containers – there I came across terms like Chroot, Cgroups, Namespaces , how the whole container ecosystem was born. I have been always been inquisitive about containers and though I haven’t really worked on containers before this talk really fascinated me to dive into the world of containers.

Then I attended a talk on What hurts people in Open Source Community . The talk helped to set my expectations right regarding contribution to a Open source project and Community.

After that I went to the Closing Keynote of the day shortly after which we went for evening snacks where we had more discussions over Coffee and Dosa – we noticed a item mentioning ‘Open Dosa’ over which we laughed a lot 😛 . And it was finally close of the day.

Day 2

I reached a little late to the venue and went straight to the talk that I didn’t want to miss. It was a Documentation BoF where speakers were discussing how to create effective documentation and tools for creating collaborative documentation. I came across User Story based documentation and tools of the trade like asciidoc and asciidoctor . I met Ladar Levison there during that session and talked with him regarding better project organisation. He gave me his business Card which mentioned Lavabit . Little did I know about him until I read this article which explained more about Lavabit and his role in Snowden’s secure email communication. But that was after this conference and I wished I could talk more about Privacy and Lavabit projects.

After that I went for lunch with Sayan, Chandan and Rayan where we chatted on lot of different stuff on open source, food and conferences. After Lunch I went to attend Sinny‘s talk on Fedora CoreOS whom Sayan introduced last day.

Finally it was nearing the end of the day. I went to attend the closing keynote by Jered Floyd and sat beside Christian Heimes from Red Hat who was sharing anecdotes from his travel experiences.

Notes from the Conference

I made few notes that I would like to share from my experience at the Conference and also as a note to me for future Conferences

  • Try to search about people you meet so that you can know more about them. You may not know everything about the person you are talking with. But actually the person can be a mine of knowledge. Ask for the person’s email/Twitter so that you can follow up on email or Twitter after the conference.
  • It’s always good to prepare some questions if you are likely to meet a person you met/knew online. You have the opportunity to talk face to face and ask about the projects the person works on. You can even do that within the conference when you are free.
  • When you attend a Talk ask good questions that can start a conversation. Usually people take a interest in following up after the Talk with you and you get to talk to more people.
  • It’s always good to be Speaker at the Conference. That way there is higher chances that you can start a conversation with people you don’t know and meeting for the first time. This is something that I really need to work on and hopefully I will be able to submit a talk in the next Conference I attend.
  • When going for Lunch tag along with a group so that you get meet more people. If you are an introvert this really works well as you can meet friends of friends and you can interact much easily!

And yes don’t forget to take pictures 🙂 It really bring memories. It may sound weird but this is something I really forget every time I meet up with people and later wait for Conference photos.

A New City, A New Beginning

Two and a half months earlier I shifted to Bangalore after a 2 years stint at my first Company at Hyderabad. I was looking for new opportunities and started appearing in interviews when I hit the hard realisation that what I learned till now was not enough. I need to self-learn more and interact more with people and learn from stories that people experienced. I landed up a job at a EDA based firm at Bangalore and decided to move in there.

Leaving the city, but not the memories !

Time really flies and it had been already 2 years working in the Software Industry. I met with a lot of people, made friends and shared some good experiences. I became aware of the meetup culture, realised writing code is not the only way that makes you a good developer, realised that communication is a key factor in conveying your ideas.

I became part of the HydPy Community and organised 2 conferences – PyConf Hyderabad 2017 and PyCon India 2018 . I met some like minded people passionate about technology and flourishing the Python Community at Hyderabad. I still am a part of the Community and hope to continue doing so.

I also attended the dgplug Summer Training in 2018, made many friends there as well whom I keep meeting during conferences. They are a really amazing community and there is always something to learn from each of the IRC conversations.

I interacted with a lot of people from Python and open source Community. Going to Conferences and meetup is really a good way to interact with some awesome people and sharing their knowledge. But it’s also true that you really need to work on something tangible and develop your skills. Then only you can move forward and also put the experiences to use.

I have been living in the heart of the fast growing city at Hyderabad. It’s been a nice experience here as a whole (except from traffic on rainy days!). I explored to places around in the initial days. Some significant events happened during my stay – the GES 2018 was held, Hyderabad metro rail was inaugurated, the first IKEA store in India opened up . I Spent my first Durga puja away from home. Realised that Kolkata Biryani is still better than Hyderabadi Biryani. Saw more buildings, Tech parks and flyovers being constructed in Rapid phase. Got to experience the hot summer and pleasant winters. All together it had been a good time in the city of Nizams. Hope to come again here someday !

The New Phase, What Next ?

It’s been 2.5 months while I came here at Bangalore. It’s called the Garden City of India because of the amount of greenery it has. But I would rather call it a City of Traffic ! Dealing with traffic can be terrifying here. Nevertheless there is a lot of greenery left and I see palm trees here and there quite often. The area I live is booming with an array of multinational companies, I also observed more number of semiconductor based companies. Rightly calling it “Silicon Valley of India”. It’s pretty early to state my experience in this city so maybe I will write about it in the next phase 🙂

So what next ? I want to make my time here more productive and also develop some good habits that I have been lingering about. Get into the habit of writing, take time out for more self-learning, contribute to open source more often, interact with more people, put the experiences to use as much as I can, get more exercise and cut laziness. In fact I got me a cycle and have been doing my daily commute with it! I know now it’s more talk than work now. But I want this blog post as a reminder so that whenever I wander off I can come to this page and find what I need to do! And keep myself prepared for the next phase.

Adding Print Preview Support to CEF

Chromium Embedded Framework (CEF) is a framework for embedding Chromium-based browsers in other applications. Chromium itself isn’t a extensible library. It helps you to embed a chromium browser in a native desktop application. The project was started by Marshall Greenblatt in 2009 as an open source project and since then it has been used by a number of popular applications like Spotify, Amazon Music, Unreal Engine, Adobe Acrobat, Steam and many more (full list can be found in this wiki). CEF is supported for Windows, Linux and macOS platforms.

There are 2 versions of CEF – CEF1 and CEF3. CEF1 is a single process implementation based in Chrome Webkit API. It’s no longer supported or maintained. CEF3 is a multiprocess implementation based on Chromium Content API and has performance similar to Google Chrome.

Preface

The purpose of writing this article is to document the work that I did while working on the CEF project. The major focus is on the Print Preview addition to CEF upstream project where I worked on.

For the past 1 year I have been working on CEF as a part of my day job. Initially the work was to maintain updates of CEF with every Chromium version released. We had a fork of the CEF open source project in which we applied some extra patches as per the requirements of the custom desktop application that used it. Building CEF was quite similar to building Chromium. It had a build script which was mostly used from downloading the code, building and packaging the binaries. Everything is documented here in this link . Upgrading the CEF was quite some task since building CEF took a lot of time and resources (a lot of CPU cores and a lot of Memory) and since CEF was based out of chromium I had to skim through parts of chromium code. Chromium has a nice developer documentation and a good code search engine that eased a lot of things. But owing to the fact that chromium has a huge codebase the documentation was outdated in few areas.

Feature Description

The interesting part of the CEF project came when I was handed over a work of a missing piece in CEF. Chromium supports in browser print preview where you can preview pages before printing, similar to one shown in the picture below.

CEF didn’t support this feature and had the legacy print menu where you cannot preview the pages to be printed.

This meant applications that used CEF couldn’t support print preview within them. The task was to make print preview available in CEF.

Initial work

The work started with CEF version 3112 (supported chromium v60) and was in a working state in CEF 3239 (supported chromium v63) in our CEF fork. Then the change was supported only in Windows since our desktop application that used it was a Windows only application. I was handed over the work in CEF 3325 (supported chromium v65) where the following specs already existed in the Print Preview patch. The relevant blocks of code is available in the CEF source code now.

  • enable_service_discovery is disabled now
  • CefPrintViewManager::PrintPreviewNow() will handle print-preview
  • CefPrintViewManager will not handle the print now. It handles the PrintToPdf function exposed through browserHost. Also, it listens to two print preview message PrintHostMsg_RequestPrintPreview, PrintHostMsg_ShowScriptedPrintPreview and generate the browserInfo for print preview dialog
  • Define interfaces from web_modal::WebContentsModalDialogManagerDelegate and web_modal::WebContentsModalDialogHost required by the constrained window of print preview dialog
  • Define platform specific definitions of GetDialogPosition() and GetMaximumDialogSize() used by browser platform_delegate
  • Register Profile preferences required for printing
  • Add ConstrainerWindowsViewsClient class which is called in browser_main.cc
  • CefPrintViewManager::InitializePrintPreview() initializes the print preview dialog which further calls PrintPreviewHelper::Initialize() which generates the browser_info required by print preview
  • Remove CefPrintViewManagerBase and its associated methods from CefPrintViewManager. Those methods are redundant after the print preview changes
  • Check for switches::kDisablePrintPreview in CefPrintRenderFrameHelperDelegate::IsPrintPreviewEnabled() to determine whether print preview is enabled
  • Add chromium patch to fix errors in debug build

My Contribution to CEF Print Preview

After I took over the CEF print preview work I did a number of changes to print preview, specs of which is documented below

  • Add print_preview_resources.pak in BUILD.gn to fix blank screen error which came from chromium v72 onwards because of updated print preview UI
  • Add PrintPreviewEnabled() to extensions_util
  • Add switches::kDisablePrintPreview to kSwitchNames in CefContentBrowserClient::AppendExtraCommandLineSwitches() to disable print preview on using --disable-print-preview command line switch
  • Remove print_header_footer_1478_1565 chromium patch since it’s no longer required to disable print preview by default
  • Add WebContentsDialogHelper() to wrap web_modal::WebContentsModalDialogManagerDelegate and web_modal::WebContentsModalDialogHost interface methods. Move it in a separate header and cc file
  • Add support for disabling print preview via chromium preferences
  • Disable print preview for OSR mode
    • For ui print this is done by using CefBrowserHostImpl::IsWindowless() method
    • For scripted print this is done by passing the is_windowless parameter in CefPrintRenderFrameHelperDelegate object from CefContentRendererClient::MaybeCreateBrowser() method in content_renderer_client.cc
  • Fix DownloadPrefs::FromBrowserContext for CEF use case. Add GetDownloadPrefs() to CefBrowserContext and use that to get the download_prefs in chromium
  • Add extra_info param to CreatePopupBrowserInfo()
  • Fix MacOS build. Add GetNativeView() method for MacOS platform delegate and update GetMaximumDialogSize() method
  • Disable print preview for MacOS (in extensions::PrintPreviewEnabled()) since the print dialog crashes on print
  • Disable print preview if pdf extension is not enabled
  • Use CEF file chooser instead of chromium while saving to pdf in print preview. Add ShowCefSaveAsDialog() and SaveAsDialogDismissed() to PdfPrinterHandler.

Challenges faced

Integrating print preview was a big and non-trivial change in CEF since not only it needed good understanding of the printing code in chromium but also the print preview feature was getting constant updates from Chromium. The code was constantly changing with every chromium version released and the print preview chromium documentation was outdated.

CEF3 has a multiprocess architecture similar to chromium’s documented here . There is a main browser process and multiple renderer processes. Debugging multiprocess applications can be trickier. I used Visual Studio which made things a bit easier as it has the Child process Debugging power tool which is a extension that would automatically attach Child processes and helped to debug into the child processes whenever they spawned up.

The chromium v72 version introduced a new print preview UI which broke the renderer, we got a blank screen in print previw. It took weeks to figure out what was wrong. Finally it came out that a pak file was missing which needed to be included in BUILD.gn. I had to spend multiple debugging session with my team to figure that out.

Also it had to be supported for all platforms (Windows, Linux, macOS) to qualify to be merged to the CEF upstream repo. Each platform had a different way of rendering dialogs. Though the windows support was working the Linux and MacOS weren’t supported in the changes yet. I added the support for Linux platform after building CEF in a linux VM. The MacOS support finally didn’t work out and we had to keep using legacy print for Mac platform. Though I needed to ensure the change built fine in Mac, so I had to build it for Mac as well (I was given a separate Mac machine just because of this since Mac doesn’t ship VM images) and in fact the change broke the MacOS build so the issues had to be fixed.

Conclusion

Even after all these changes the functionality broke after a architectural change was made in CEF in version 3770 (supported chromium v75) in this commit which rendered blank screen during print preview. Marshall took over the work from there and made a number of changes in the patch which can be seen in this PR chromiumembedded/cef#126 . The change was added manually in master revision 1669c0a on 20th July. It will be supported from the next CEF version (supported chromium v77). The current implementation supports print preview in Windows and Linux platforms via a --enable-print-preview flag.

Overall it has been a good experience working in the project and I got to know a lot about chromium itself. This was my first major contribution in a C++ based project. It helped me to understand how a browser works under the hood, how a browser renders web pages and processes javascript. I hope to carry forward this knowledge in some similar browser based project in future.

Understanding DNS

In this post I will explain How DNS Works in my own words.

DNS stands for Domain Name System. It is way for naming the Address of computers or resources in a Network. All of the computers in a network is associated with an IP Address, like IP address of google.com is something like 172.217.163.110. Now its tough for a person to remember such big numbers. So we have a simple and more human friendly way of naming these computers which is done by DNS.

So now how does the DNS work ? When we type in a website name in a browser the IP address of the server mapped to the name needs to be found out to display the web page. So there is a process by which this IP address is searched. Let’s understand what are the steps that are carried out for  finding the address.

First, Ask the Browser

Whenever we type in a website name in a browser the browser searches its cache to check if the IP address mapped to the website is present.

Then, Ask the OS

If the Browser doesn’t have the address it then asks the OS to check if it has the address.

Ask the Resolver

If the OS doesn’t have the address it points to the IP address of the Resolver server that will Resolve the IP address of the website. It is the role of the Resolver to find the IP address of the website and bring it back to the OS. These are usually the Servers provided by the ISPs serving the Internet. If you do a cat /etc/resolv.conf in a Linux Machine you will get an output similar to

# Generated by NetworkManager
nameserver 202.88.174.6
nameserver 202.88.174.8

These are the IP addresses of the Resolvers that are responsible for finding the IP address of  website when a request comes to it. So it first checks its cache to see if it has the IP address of the website requested. If it doesn’t find the IP address it then goes to the Root to find the same.

Ask the Root

The Root server knows the addresses of the Top Level Domain (TLD) Server for the website. There are total 13 Root Servers spread all over the world. Well that doesn’t mean there are only 13 servers. Basically it means there are 13 unique names for the server. Each one is distributed over multiple servers to handle the load. The Resolver gets the address of the TLD Server from the Root and goes there to find the IP address of the website. Each time the Resolver gets the address it saves it to its memory.

Ask the TLD Server

The Top Level Domain is the .com part in google.com . Similar to that there can be various Top Level Domains such as .org, .gov, net, .edu etc. Also there are country specific  domains like .in, .us. .jp etc. The Root server knows the addresses of these TLD servers. The TLD then gives the address of Authoritative the Nameservers for the website domain.

Ask the Authoritative Nameservers (DNS Servers)

The Authoritative Nameservers are the one that contains the actual address of the website. Their names are similar to ns1.google.com, ns2.google.com etc. These are often simply called the DNS Servers as they contain the records of the address corresponding to a specific website name. Whenever you purchase a Domain the Domain Registrar send the name of these DNS Servers to the the TLDs. That way a TLD can say which DNS Server contains the address of a website. The DNS Server gives the Resolver the IP address of the website.

You can find the names of the DNS Servers of a website and the website IP address using the dig command. Here is a sample output of dig command.

[ananyo@localhost ~]$ dig google.com

; <<>> DiG 9.11.4-RedHat-9.11.4-1.fc28 <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33490
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 9

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 8114e77595b2cc097e7725a95b59fdf1b2d1c4d480039c49 (good)
;; QUESTION SECTION:
;google.com. IN A

;; ANSWER SECTION:
google.com. 40 IN A 172.217.163.78

;; AUTHORITY SECTION:
google.com. 31267 IN NS ns3.google.com.
google.com. 31267 IN NS ns2.google.com.
google.com. 31267 IN NS ns4.google.com.
google.com. 31267 IN NS ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com. 206437 IN A 216.239.32.10
ns2.google.com. 210049 IN A 216.239.34.10
ns3.google.com. 210049 IN A 216.239.36.10
ns4.google.com. 210049 IN A 216.239.38.10
ns1.google.com. 210874 IN AAAA 2001:4860:4802:32::a
ns2.google.com. 341654 IN AAAA 2001:4860:4802:34::a
ns3.google.com. 57401 IN AAAA 2001:4860:4802:36::a
ns4.google.com. 304702 IN AAAA 2001:4860:4802:38::a

;; Query time: 35 msec
;; SERVER: 202.88.174.6#53(202.88.174.6)
;; WHEN: Thu Jul 26 22:29:29 IST 2018
;; MSG SIZE rcvd: 331

Finally, return it to the OS, then Browser

The Resolver finally gives back the IP address of the website to the OS which then caches it for future requests. The OS then gives it back to the Browser which sends the request to the IP address and serves the Page. So if you enter the IP address of the google.com in the browser that we got from dig command (172.217.163.78) it will point to the same page.

The Best part of this is the entire things just takes few seconds to complete!

 

Understanding the Linux Command Line

For quite some time I was thinking of learning Bash scripting. Thanks to dgplug Summer Training I have taken the first step to Learn the Linux Command line. We are using the LYM (Linux Command line for You and Me) book in the classes to learn the basic Linux commands and understand the GNU/Linux system.

Why should you learn command line ?

  • The command line is a very powerful tool for a GNU/Linux system. You can do anything from the command line starting from editing a text file to shutting down your system.
  • Anyone who wants to understand the GNU/Linux system must have a understanding of the command line.
  • Automating day to day to day stuff becomes easier if you know command line. This is especially indispensable for Sys Admins and Network Managers.

There has been already few classes in dgplug Summer Training where we were asked to read few chapters from the book and ask doubts regarding it. I use Fedora 28 as my Linux distribution where I use the gnome-terminal for running the shell commands.

I have learnt quite many things till now. I am on the seventh chapter of the book now which is “Linux Services“. Slowly I have been sinking all the commands to my mind and I hope to make use use of them to make my life easier with my Linux machine. The book is well written for a beginner who is just starting with Linux command line on the other hand it has also references to some important topics like FHS .

Thanks to the authors for their efforts to write this. I  hope I can contribute to it at some point 🙂