An Introduction to Git for Hardware with Altium Designer
Introduction
A few years ago, I temporarily transitioned from working in a hip hyper-caffeinated startup environment to an engineering job in a more well-seasoned and experienced company. I then realised the binomial connection between technical culture and tools. How we think and how we work is shaped by the tools available to us, as our needs and customs influence the tools we choose.
The new company where I worked wasn't accustomed to 3D printers, in-house bug tracking software, or even a good CMS. It was all rather old-fashioned. This had a significant influence on what my colleagues and I created daily.
An example would include the past couple of decades, where the industry has transitioned from TO220 packages to D2PAK. At the same time, engineers have equipped themselves with high peak-power soldering irons, such as those manufactured by JBC.
Would a young engineer accessing a well-equipped lab even consider any popular IC in a TO220 in this decade? I don't think so. It's so much easier to work with D2PAKs and not fumble around with screws, washers, and insulating foil, provided you have a last-generation soldering iron. This simple change can steer an engineer towards designing flatter boards, which can often lead to more aesthetically pleasing products by modern standards.
Git is a rare example of a tool that has flipped a whole industry on its head. Ten years ago, software engineering managers would have considered madmen to adopt a move fast and break things approach. Git for hardware and PCB design has made it possible by enabling revision tracking, version control, and rolling back design changes. Developers are now assured their efforts to open-source projects can always be attributed and verified, thanks to a feature called Git blame. A decade ago, being recognised for your contribution to open source projects was left to politics. These are all examples of changes we can thank Git for.
While by its own nature, the electronic industry moves slower than software, many of the innovations are trickling down to our day-to-day work. Altium Designer®, with the introduction of both Altium 365® and Concord Pro™ this year, has led the way in the industry, with other vital players struggling to keep up, sometimes with features released over a decade ago. Git for hardware and PCB design is one of the technologies powering that change.
What is Git?
Very simply, Git is a version control system (VCS). It's a piece of software (including the underlying protocols and data formats) used by software developers to track and manage code changes. If you are a software developer working in this decade, you don't duplicate folders on your desktop to try things out, you most likely use a VCS based on Git.
Although Git is massively popular for tracking code changes in software development, it can be used to track changes to any set of files. These files do not need to contain code, they can be your PCB design files, design documentation, PCB manufacturing files, and any other files you'll need for your project. Git for hardware is a natural extension of the Git ecosystem to mechanical design, PCB design, firmware, and much more.
Git is freely available for commercial use. It is open-source and distributed under the GNU General public license. Each Git directory is a separate entity and is, itself, a repository containing a complete history of its items. Every file placed in a Git repository is fully trackable down to each bit, by whom, and when. Git repositories do not require network access, with each repository being completely independent of the server(s) that take the name of remote(s).
It should come as no surprise, then, that it's currently the world's most popular VCS. Most market share analyses show Git over 75%, and the most popular alternative, SVN, has been in decline since 2012. The number of job vacancies requiring SVN (A legacy VCS, also supported by Altium Designer) is also in decline while Git has been gaining popularity.
History of Git
Git was created and written in 2005 by Linus Torvalds, the man, the legend, the creator and developer of the Linux kernel, to keep track of the kernel's very own development. The Linux community had been granted free use of a commercial piece of software called BitKeeper. In April 2005, Bitkeeper's author withdrew the license after a prominent member of the Linux team and inventor of the Samba file server, Andrew Tridgell, began working on an open-source client based on the (allegedly) reverse-engineered the BitKeeper protocol. The BitKeeper license expressly forbids reverse-engineering.
Thus, Linus Torvalds decided to create his own version control system, not-so-loosely inspired by BitKeeper, as none of the remaining alternatives was anything close to fulfilling his requirements.
Torvalds decided that the new software would be very different from the popular CVS (Concurrent Version Systems) in use at the time. He realised that current systems could take a long time to apply patches, and as he needed to apply hundreds of patches at once when synchronising with his team, their performance was far from acceptable. He came up with a series of requirements:
- A distributed workflow similar to what BitKeeper enabled. The user should be able to work offline and synchronise later.
- Protected against accidents such as data corruption
- Secure against malicious attacks
- Able to compute patches in less than two seconds
Work on writing Git started at the beginning of April 2005. On June 16th, 2005, version 2.6.12 of the Linux kernel, the reason why the software was needed in a hurry, was released. Development and maintenance of Git were then handed over to Junio Amano, who contributed and still contributes to its development, and is widely credited with making the software easy to use; Git 1.0 was released in December 2005.
Naming Convention
Why Git? A strange name, to say the least! As most people in the UK know, the term is often given to someone who is a bit cheeky or, according to the Oxford Online Dictionary, "an unpleasant or contemptible person". Additionally reported meanings are "fool" (18th to 19th-century slang (UK)) or "bastard child" (mid 18th to 19th-century slang (US)), both of which would fit quite poetically into its myth of the lone genius hermit hiding away to create a work of art that would change the world.
Torvalds has given several reasons for naming his system "Git", to be chosen based on what the user is feeling on the day, or probably how he was feeling at different times when writing the system! He often describes it as "the stupid content tracker" on the official documentation, and the definition of Git as being:
- "A random combination of 3 letters that can't be pronounced".
- A mispronunciation of "get"!
- Global information tracker if it works according to plan
- Stupid, contemptible, and despicable when it doesn't.
Oh, programmer's humour.
Disadvantages
Git is not perfect, however, and it has some disadvantages. The hard-to-grasp data structure and the weird nomenclature are without a doubt among them. This includes Git for hardware projects, where the same file structure and operations are enforced.
Cherry-picking, Checkout, Index, Clone, Push, Stash, Pull/Pull Request, Tag, Upstream, Fork, Rebase, Origin, Fetch, and HEAD (always written in full uppercase, I have no clue why) are among some of the weirdest terms you can expect to find in the software world.
It can be hard to understand how to set up a repository as server-side software, and to understand the relationship between local and remote repositories alongside the operations to keep them in sync. Git tends to be much more complex than SVN to learn and use, partially due to it being much more powerful and efficient.
Luckily, Altium Designer and Concord Pro take care of most of these issues. While we have full access to the power of Git through the command line, while the user interface and strict integration of Concord Pro make it easy and intuitive to operate. At the same time, Altium 365 takes care of all the server-side troubles.
How Does Git for Hardware Work?
Git can come across as... very weird! The nomenclature, mainly, reflects a workflow that differs substantially from the classic copy-pasting, zipping and emails many engineers are used to.
Branching (and Merging)
The branching model is perhaps the most popular feature that separates Git from another VCS like SVN. Git can have multiple branches, both local and remote. Like the branch of a tree bifurcate out of the trunk or from each other, so Git branches sprout from other branches. The "trunk" of the tree, or the main branch, is called master. The branches can be easily created, merged, and deleted. Here's how these operations work:
- Each branch is independent, and when working remotely, you don't have to step on anybody's feet. You can even have multiple branches yourself, each branch containing a slightly different variation of your own software or hardware design, and they can be switched within the same directory without having to close and re-open files manually.
- In the software world, the rule of thumb is to have a production branch, called the master, and a second work-in-progress branch called develop and as many small branches as needed for new features and fixes. The same approach can be taken with hardware projects. In fact, there are many GitHub repositories with PCB designs and other projects specifically for hardware.
- Not all branches have to be merged into the master branch. Often developers figure out that the new feature isn't quite a spark of genius, and the branch can simply be deleted when no longer required.
[Nerd mode ON]
So how does this clever feature work? A branch is essentially a pointer to a commit. A commit is a set of file changes, additions, or removals pushed into the repository. The commit has a 40-character cryptographic SHA-1 checksum that gets written to a file. Each commit also includes a pointer to the parent commit it originated from.
There are many additional intermediate steps, for example, files are converted into checksummed binary blobs and organised in directories through a binary tree. The checksum of the tree is computed as well. Since everything is hashed cryptographically, there is no way of altering (or corrupting) the data or history without changing the hash of the last commit. The cryptographic hashing makes Git's history somewhat permanent, so be polite when writing commit messages!
[Nerd mode OFF]
Workflows in Git for Hardware
Thanks to the distributed nature of Git for hardware and the advanced branching system, as well as a host of other features, users are free to adopt any workflow.
Among the most popular, the *Centralised Workflow* model is one often used when people who have experience with centralised version control system start using Git (which is *decentralised*) for the first time. The *Centralised Workflow* relies almost exclusively on the master branch, where all commits are pushed to and fetched from, making Git mimic SVN's behaviour and remote filesystems.
The Feature Branching workflow is an evolution of the *Centralised Workflow*. The development work is done on separate branches that are then merged into a master. I am an enthusiastic proponent of this model in electronic engineering, and I am anxiously waiting for Altium to announce their branch support to take advantage of it. Examples of feature branches could be “fix_current_generator_oscillation”, “upgrade_microcontroller”, “lower_power_consumption”, or “reduce_thermal_drift”.
In the GitFlow workflow, perhaps the most complex among the popular workflows, the master branch only contains complete design releases, you can think of it as board_v_1.0, board_v_1.1, etc. Development is made on a separate branch called develop, and feature and fixes stem from the develop branch. Only develop can be merged into master once it is ready.
Decentralisation and Speed
Git is faster than other version control systems for several reasons. Each user can clone the original repository, and commits can be made regularly to local branches and sent to the remote less frequently. Other VCSs that are not as decentralised are limited by the remote server's capacity, which has to slow down considerably to satisfy all their requests.
This local-first approach is particularly crucial in the electronic industry, as files can be rather large. It's not uncommon for a PCB design to be tens of megabyte in size, especially with hundreds of 3D bodies. Source code files, on the other end, tend to be a few hundreds of KByte at most.
In the last company I worked for, we had an SVN repository hosted in the main offices, accessible through a VPN, where we would store the project files and documentation. It would take forever to do any operation, and our limited internet connection was frequently clogged by all the requests to manage the thousands of files.
Git is also written in the C language, meaning its overhead is minimal compared to other high-level languages. Depending on the operation, Git can be from a few times to hundreds of times faster than SVN.
The decentralised, offline-first approach also makes Git much lighter on the network. Even if your company does not have access to broadband, you can push the data at lunchtime or after work, with no loss in performance in the day-to-day work.
On Concord Pro, you can enjoy all the benefits of Altium 365 when you have access to an internet connection, then roam your Altium Designer license and continue working offline.
Staging Areas
When working with Git, it is essential to realise that files pass through three stages before they can be indeed considered to be under version control:
- Untracked
- Staged
- Committed
Untracked is when the file exists on disk, but is out of the version control system. The Untracked file can then be staged, which means it has been added to the version control system but not yet committed. At this point, the staged changes can be committed. The staging system is used to prepare for the commit, but the feature is used mostly in command-line operation.
When using Git through Altium, thanks to the graphical user interface that simplifies operation, the staging approach is applied automatically in the background when you save the changes to the server.
Creating a Repository
Repositories are automatically created on the server-side when you create a new project. In the window below, I'm creating a new PCB project from a template within my Fermium LTD workspace. As soon as I create the project, it will be accessible in my Altium 365 workspace, and the platform will automatically create a Git repository for the new project.
_0.png)
Initial Repository Configuration
The repositories created with Concord Pro are automatically configured, so only the essential files are already committed to the project's Git repository, while local backups and LOG files are not. It would typically be necessary to both configure a file called ".Gitignore" appropriately and take care of not committing unnecessary files, but Concord Pro takes care of that.
Setting up Credentials
To access Git repositories, one would generally need to configure SSH keys and user credentials on both the server-side and the client-side. Concord Pro takes care of this automatically when a new user is added.
.png)
Cloning Repositories
Cloning is the process through which Git replicates the remote repository into a local copy. Manually cloning large repositories with binary files, such as PcbDoc and SchDoc files, would normally require expert-level command-line operation of Git. Altium Designer and Concord Pro automatically clone the repository to your local machine in the background when you open a remote project.
Resolving Conflicts and Comparing Changes
Concord pro allows you to compare changes between different revisions, both local and remote, through the storage manager panel. In the following example, I have added a text note object and committed the changes locally, but did not push them to the remote.
.png)
This gives us the complete functionality needed in a Git for hardware platform.
Conclusions
Git is a formidable tool, and git for hardware gives PCB designers a comprehensive workflow for version control, sharing, and revision management. This popular system has helped shape the culture of modern programmers. Now with Altium Designer® and the Altium 365® platform, designers can access Git features for PCB design.
You can start a trial of Altium 365 today, loaded with example projects, to fully experience electronic development 21st-century style. Have more questions? Talk to an expert at Altium today.
