The coding weeks are over!

So people, the coding weeks are over. This post is for a reference to the work done by me during this period highlighting the goals achieved and the outstanding work.

The task was to develop Gigaibit Ethernet Media Access Controller(GEMAC) (the MAC Sublayer) in accordance with IEEE 802.3 2005 standard using MyHDL. The aim was to test and help in the development of MyHDL 1.0dev, also demonstrating its use to the other developers.

In brief, work done includes developing Management Block and Core Blocks, i.e., Transmitter and Receiver Engine with Address Filter and Flow Control. Work left includes developing the interfacing blocks, i.e., FIFOs (Rx and Tx) and GMII.

Post Mid Term I started implementing core blocks. Midway I realised that I would be better off using Finite State Machines to implement these, which led me to rewriting the whole blocks. Currently, I am looking towards implementing the interfacing blocks; FIFOs (for which i shall try and use already developed blocks by other developers) and GMII(depends on the PHY, I will be using the one that comes on Zedboard, meaning i would be developing RGMII).

Tests for each blocks were developed using Pytest. Seperate tests were developed to test each unique feature and ensure its working. Also convertibility tests were developed to test the validilty of the converted verilog code which shall be used for hardware testing in the end.

Main Repo :

Links to PRs:
1.Implemented Modular Base:
2.Implemented Management Module:
3.Implemented Transmit Engine:
4.Implemented Receive Engine:

My main focus after I am done with this is to make this code approachable by other developers by providing various good examples of using the library.


Finite State Machines

Well Its been tough couple of weeks. Proceedings in my university has caused me to slow down a bit. But i have been making progress. I was working on RxEngine Block when things got too complex and I decided to take a step back and refer to use of Finite State Machines(FSMs) to develop much more simple and readable code. As it turns out i ended up rewriting the TxEngine Block from scratch as well. In midst of all this the file system in my local repo got too clumsy as i had multiple versions of RxEngine implementation and planned to wait out for the final revision of the blocks to avoid problems with commits later on while rebasing. I shall push the latest code in a day or two for review.

While implementing TxEngine block using FSMs I added the underrun functionality which was remaining in the previous implementation. Also I did a rough implementation of Flow Control Block which accepts request from client to send pause control frames and triggers the TxEngine for the same.

Also i had discussion about how to provide clocks to sub-blocks and handling the reset with Josy, one of the mentors, who suggested providing clocks to sub-blocks directly in the top block as opposed to relaying them through the sub-blocks. A good reason that i can think of to support it is that, if your system is a bit big and complex it might cause problems in simulation. I shall discuss more about it in detail in upcoming blocks.

Started Receive Engine!

Its been a long time since my last post!(2 weeks phew)! Sorry for the slump. Anyways During the period i successfully merged Transmit Engine after mentor’s review. I later realised that i missed adding functionality of client underrun used to corrupt current frame transmission. I shall make sure to add that in next merge.

Next I started looking towards GMII, which partly stalled my work cause i was unable to clearly understand what I have to do for that. So I decided to move on and complete Receive Engine with Address Filter First. Till now i have finished receiving the destination address from the data stream and filtering using the address table by matching it against frame’s destination address. If there is any match, the receiver starts forwarding the stream to client side, otherwise just ignores it.

Next i look forward to add error check functionalities to be able to assert Good/Bad Frame at the end of the transmission.

CRC32 : Transmit Engine

Completed first draft of implementation of Transmit Engine. The implementation was fairly straightforward barring the calculation of CRC32(Cyclic Redundancy Check) for Frame Check Sequence.

It stalled me for a day or two requiring patience while reading and understanding the type of implementations. A very painless tutorial for understanding crc32 and its implementation from ground-up can be found here. This implementation in C also helped

Now I have generated pull request for code review.

Started Transmit Engine!

Yay readers, good news. I got through the mid-terms and received the payment. Feels good!

About the project. I started of with implementation of transmit engine. Sweeping changes had to be made in the the interfaces of the sub-blocks. Notable changes:

  • Removal of Client Sub-block and interfacing the FIFOs directly with Engine.
  • Addition of intrafaces, i.e., interfaces between the sub-blocks.
  • Moving the configregs(Configuration Registers) and addrtable(Address Table) out of management block to main gemac block to improve its scope to other subblocks. Now its accessed by management block through ports.

As a result of changing the ports of management block i had to edit the test_management to reflect the change. I had independent instantiation of the management block in every test which was redundant. I then looked up into pytest fixtures which enabled me to have a common function which would be run before every test thus removing the redundancy. It provides convenience to change the block port definitions in future if needed.

Now i am working on implementing its features. A little about Transmit Engine:

“Accepts Ethernet frame data from the Client Transmitter interface,
adds preamble to the start of the frame, add padding bytes and frame
check sequence. It ensures that the inter-frame spacing between successive
frames is at least the minimum specified. The frame is then converted
into a format that is compatible with the GMII and sent to the GMII Block.”

GSoC: Mid-Term Summary

Well four weeks of GSoC is over, and its time for the mid-terms summary and replanning.

Mid-Term Summary:

  • Studied about MACs and their working. Chose Xilinx User Guids 144 (1- GEMAC) as interface guide and reference verilog design as features guide.
  • Completed setup of main repo providing the modular base for further development.
  • Implemented Management Sub-block.
  • Setup the repo with travis-ci build, landscape linting, coveralls.

So comparing with the timeline in the proposal i have achieved targets of first four weeks switching the management and tx engine modules.

Further Plans:

  • Take three other sub-blocks mentioned in the proposal timeline and try and implement them in a week each (rather than two weeks as proposed).
  • Implement wrapper blocks including FIFO and client in the next week.
  • In the remaining weeks, hardware testing, refactoring code if necessary, setup of readthedocs shall be done.

Maintain A Clean History!

Finally I have completed and merged the management module. Last time I posted, things i needed to be able to merge was to add the doc-strings, setup coveralls, resolve conflicts with master branch(rebase).

Adding Doc-Strings was the easiest but still took time as it gets a little boring(duh!). I used this example provided by my mentor as reference.

Now came time to do a coveralls setup, which i must say i a little more complex compared to the others. I really got a lot of help from referencing an already setup repo test_jpeg on which a fellow GSoCer is currently working. It got a little tricky in between as i stumbled upon the type of end-of-line character problem. Before this i didn’t even know that even an “type of enter” can cause problem in running scripts. It consumed my one whole day. It bugged me when i was trying to edit it in notepad on Windows. This post later helped get me over it. More on coveralls setup on my next post!

Next Rebasing and resolving conflicts my dev branch compared to master branch. When i started my master branch was a few commits ahead (setting up of badges) and thus was having conflicts. Also Rebase was required as my mentors suggested to maintain clean history in the main branch. It took me lot of experiments to finally understand the way to go for rebasing my branches. The structure of my repo:

  • origin/master
  • origin/dev
  • dev

So i have a local dev branch in which i develop my code and constantly push to remote origin/dev branch for code reviews by my mentor. This leads to lot of commits containing lot of small changes and resolving silly issues. But when i make a pull request and merge onto origin/master branch I wish to have cleaner commit history.

So doing an interactive rebase helps to modify that history using pick(Keep the commit), squash(Merge onto previous while editing the commit description), fixup(Merge onto previous keeping the previous commit description intact). Understanding this required me doing lot of experiments with my branch which is dangerous. So I had made a copy of my dev branch, which i suggest you do right now before continuing.

To rebase your local branch onto origin/master branch use “git interactive -i <base branch>“. Warning, avoid moving the commits up or down if the are working on the same file. This may cause conflicts. Once it starts, Resolving conflicts is lot of pain because it triggers other conflicts as well if not done properly.

After rebasing come the trickier part. Your local branch has brand new rebase commits and your remote has old commits. You need to use “git push –force”. It will overwrite the commits on remote branch after which you can generate a pull request onto origin/master. Don’t do it if there are other branches based on this branch In that case directly merge onto master, downside being you wont get to be able to make pull request on which is essential for code discussions.

After all this my code was ready to merge and i got go ahead (after a day of internet cut, hate that) from my mentor to merge it. So i had finally completed second merge on to my main branch implementing the management block and setting up coveralls.