What can be learned from the OntoGraph project?

OntoGraph was introduced in my last post, OWL Ontology Graphing Program Available as Open Source. And there are a lot of interesting things in the code! Over the next weeks, I want to take time to relay my learnings, as well as to provide insights into OWL ontologies, SPARQL queries, Bootstrap, Backbone and RESTful interfaces, the Model-View-Controller and other patterns, Spring Boot, Lombok, programming Stardog, testing, Gradle builds, and much more. Some of this will be basic stuff (but hopefully useful to some of my readers) and some will be more advanced. Feel free to pick and choose, or let me know what you want to hear about!

But, first, I want to talk about our development environment ...

The precursor to OntoGraph was originally created in about 2 days to provide some basic diagrams of a customer's ontology. Hand-drawing all the classes, properties, axioms, etc. of the ontologies was too painful and error-prone. Using a tool like OntoViz with Protege was just not flexible enough, and the images were not what the customer wanted to see. The ProtegeVOWL plug-in was also not sufficient since VOWL does not diagram all the necessary constructs (I will talk more about this in a future post). In addition, the customer did not want to be tied to using Protege since they weren't ontologists. They just wanted a diagram and to be able to play around with the layout.

Well, the 2 day "quick and dirty" version worked and the customer had their diagrams. That could have been the end of the story. But, we hired an intern who needed to learn about ontologies, the Stardog triple store, SPARQL queries and lots of other things. So, we decided to use the graphing program as a learning experience. We took the initial work and decided first to just address some bugs. Then, we decided to add the ability to customize the output, which required a front-end. Then, we added support for different kinds of visualization (Graffoo, VOWL, UML). And, the program grew. We changed directions, rewrote whole sections of the program, updated our approach to the front-end at least three times, updated our approach to testing at least twice, and upgraded our infrastructure at least twice (updating the Gradle, Stardog, Javascript libraries, etc.). We put months of work into the program, definitely taking an agile approach and learning to "fail fast".

There are lessons here ... Good software takes time. There is always more to learn. Don't be afraid to take what you learn and rewrite what is problematic (as long as you have time and there are no other programming fires burning). There is always something that you can do better. And, always remember that Stack Overflow is your friend!

Well, ok then ... back to agile. For our agile environment, we used Atlassian's products - JIRA for issue tracking and managing our process (Kanban actually), integrated with a Bitbucket Git repository for version control, and Bamboo as our continuous integration environment. Since we are a small company, this was an easy and cheap solution ($10 for each product). In addition, when we decided to get serious about releasing the code as open-source, we also decided to incoporate SonarQube into our continuous integration environment.

As someone who either spent too much or too little time on code reviews, SonarQube was great! Per Wikipedia, it provides "continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells and security vulnerabilities" (http://en.wikipedia.org/wiki/SonarQube). And, it does this for 20+ programming languages (but we only needed Java, JavaScript and CSS). This took a lot of the pain out of code reviews. I focused on whether the method and property names were understandable, if the code seemed reasonable and was somewhat efficient, and things like that. SonarQube took care of finding problems related to bad practice, lack of efficiency, and errors (such as not initializing a variable). In addition, SonarQube would complain if you nested if/while/for/switch/try statements too deeply, or implemented methods with too many parameters or that were too complex. In reality, SonarQube was tougher on my code than any team review that I had experienced in the past.

Now, you can make things easier on yourself and change the defaults in the SonarQube rules. For example, you can allow a complexity of 30 instead of 15, or allow nesting of if/while/... past 3 levels. But, we didn't do that for OntoGraph. We figured that we would keep the defaults and fix most of the problems (or, we would eventually fix them). There are some "issues" that are just false positives, and others that we have not yet addressed. If you want to find them in the OntoGraph code, just look for "//NOSONAR" and then the explanation that follows. The "//NOSONAR" comment tells SonarQube to ignore the issue for now - either it is a false positive or we acknowledge that there is a problem and are willing to accept the issue for now. I think that this is a valuable approach. Most of the existing issues in OntoGraph are complexity, and we will fix those!

Another important aspect is test coverage. When we decided to release OntoGraph as open source, we set a testing threshold of at least 80% on the back-end processing classes (so this would be GraphController.java, GraphDAO.java and all the classes in the graphmloutputs folder). All of these classes have coverage between 93.2% and 98.3%, except one. TitleAndPrefixCreation.java has a test coverage of 77.8%, with 2 (yes, 2) uncovered lines. Those lines throw an IllegalAccessError exception if something tries to instantiate the class (which should not be done since the class contains only 1 static method). Oh well, we decided that this was definitely good enough!

You can see SonarQube in action by downloading OntoGraph and following SonarQube's instructions for Getting Started in Two Minutes. After starting and logging into SonarQube according to the instructions, go to where you downloaded OntoGraph. Type "./gradlew sonar" or "gradlew.bat sonar" for Windows (making sure that you have installed Gradle :-). After that completes, you can see all the rules/issues, statistics and more.

P.S. Sorry for the riff on SonarQube, but I wanted to hit on some cool details. And, I will talk about how Gradle supports SonarQube in a future post. This post just got way too long!

Andrea