Code Quality at Babylon JS by Microsoft
David Catuhe from the Babylon JS team shares a few words about the code quality strategy of one of the most popular open source projects at Microsoft.
Christophe Shaw: Today I have the pleasure to have a discussion with David Catuhe. But I will let him introduce himself.
David Catuhe: Hello I am David Catuhe, I am working at Microsoft, leading the Babylon JS team. It’s a 3D engine that I created before joining Microsoft. We are using it to render silver out stuff including sharepoint spaces, powerpoint on the web also rendering the reaction in teams. It’s a team of 12 people so far. It’s an open source project by the way and it is also available on GitHub.
Christophe Shaw: David we are here today to discuss code quality. I am very curious to know what is the approach to code quality that you have with Babylon JS and beyond that. What is your approach today and how would you define it?
David Catuhe: Babylon Js was started as a pet project, something I started in my garage so we didn’t do any testing, pure code, no documentation. Today it is more a product that is used by several companies so we had to be more serious about that. Across the 7 years of the life of Babylon JS we added a lot of coverage, we have a lot of visual testing. That is mostly the part where we are investing a lot. Visual testing is very important for us 3D engines so we have a set of images and scenes and we render these scenes and we expect them to look like the referencing imaging right. The part where we are not as good as I wish we were is unit testing. We still have a lot of missing unit tests hence the connection we did with Ponicode. Something we should talk about after I guess.
My life, my professional life is splitted into two. On the one hand I am working with Babylon JS and on the other end I am working on the web client for Microsoft Stream, part of Microsoft 365.
On that last part we have excellent code coverage, roughly 92,93% of the code is code coverage and I would love to reach that goal as well for Babylon JS.
Christophe Shaw: What kind of piloting system do you have around that? What do you check beyond code coverage? How do you implement your strategy in terms of code quality?
David Catuhe: Code quality today if you look at Babylon JS is mostly 3 main aspects. The first one is performance obviously ists a 3D engine right. We need to ensure that the engine we are building is at least as fast as the previous version so we have 150 different tests that we run on A single computer that does not change. Where we run on every single version the same set of features and we check if its frame rate, the number of frames you can redner per second is evolving in the good direction. So that's performance. The number 2 is size. Size is very important because we're on the web and the web is used in new ways. Like recently for instance we worked with Nike, because now Nike customers can design their own shoes and Nike was cautious that we have minimum side of offset because if they are in the middle of a desert with a berli 2G collection you still have to be able to configure it right. And so the size of the framework itself is very important so that’s the number 2 aspect.
And the number 3 aspect is the coverage like the number of tests, the surface of the API which is tested. As I said Babylon is used internally by Microsoft and externally by companies so we do not have the option to ship a version which is not working. That is just not acceptable hence the big push on testing.
Christophe Shaw: Can you tell us when in your career you run into a situation code quality that you will always remember.
David Catuhe: It happened many times but always with the same pattern. Somehow for some reasons when we are close to release, we release every day like just fast release but we have major marketing releases twice a year. And for these releases we change the framework number and we have marketing communication etcetera. Most of the time and I don’t know if it’s just a curse or something but everytime, 2 or 3 days before we tend to find some terrible bug thanks to the tests. That is happening all the time and I don’t want to judge anything but most of the time it is related to safari. When we run the entire suite test on all the browser and stuff I guess every single time we are doing a release we find some neat picking issues that we were not able to find before. The only way to find them was to run the entire suite of tests. Every time it is a life savior, everytime we think thank god for the test. Furthermore Babylon is open source so everybody can come and make a pull request to push some new code and we are successful in the sense that we get a lot of pull requests. Without automatic testing it would have been a nightmare. Now every person doing a PR will go through the entire run of our test so we know at least that what this person is adding to the framework will not break anything.
Christophe Shaw: Do you have requirements for the people, for the test coverage and test quality for people making PRs?
David Catuhe: Making PR so far, we are at the first stage of what I would like to do, we are making sure that they are not breaking anything. We are also asking them to document the code but there is a second stage where I would like them to provide a unit test for the specific code they are adding. That's not the case yet but I want to look forward to work with Ponicode because that’s one of the main reason I want to add support from Ponicode
Christophe Shaw: Finally we have been in the industry for a while now, 7 years on Babylon JS. You have seen an evolution in code quality in all those years in the industry. What are the tendencies that you see happening today that will impact the future.
David Catuhe: I see the same in Babylon JS and other frameworks we are working on at Microsoft at least where I am involved. There is a tendency to automate as much as we can. And that’s good, as a human it’s increasingly difficult to harness and embrace the entirety of a project and I see that at Babylon JS with PR. Ideally in a perfect world, a PR done by someone, I should have automated testing that is testing everything. I should blindly click merge. So far it is not the case, I have to review the code. And we are looking to add more unit testing and I see that on more projects internally at Microsoft. We are a lot of people working on a lot of projects so the trending I am seeing here is more and more automation.
Christophe Shaw: Thank you David and finally we are so happy to have this experiment with you and contribute to the code quality at Babylon JS