RGen 0.6.0: Large Scale Modeling

This is the official announcement of RGen Release 0.6.0! Those who checked the project on Github already saw numerous new commits spread over the last year. I was actually working a lot on RGen, mostly to support our inhouse tools, but never really had the time to do the public release. There are some cool new features which I don’t want to withhold from you. I think RGen is now a very good choice if you need to work with huge models and metamodels efficiently.

First of all, RGen got faster and more modern. It now supports Ruby 1.9 which is significantly faster than the 1.8 version. To be more precise, RGen works with all Ruby versions from 1.8.6 to 1.9.3 and has been tested on Linux and Windows.

If you use one of the built-in XML model instantiators, RGen will complain unless you have Nokogiri installed. I have switched to the defacto Ruby standard parser for all of RGen’s XML parsing needs. Nokogiri is a native Ruby extension wrapping the libxml2 library. It is fast, reliable and has a large user base.

The most distinctive new feature takes RGen’s modeling agility one step further. The new ECoreToRuby transformer allows to create Ruby metamodels from ECore models on the fly. You can think of this as a superfast and elegant shortcut for generating the metamodel to Ruby source code and then loading it back into the interpreter. Here is an example: Assume you have a metamodel in the form of an ECore model, namely a root EPackage object. Then you can do the following:

The create_module method creates and returns a new Ruby module which represents the metamodel root package. Within this module you will find all RGen metamodel classes in their respective subpackages (Ruby modules) ready for use.

I actually see two major usecases for the ECoreToRuby transformer. First, this allows to keep a common (maybe standardized) metamodel in some central place and then modify or extend it depending on the need of a specific tool or user configuration. For example you could have several different tools working on the same metamodel but each of them bringing their own extension metamodel and attaching it to the base one at startup time.

The second usecase is performance optimization. Loading a large metamodel with the traditional method, i.e. require the Ruby metamodel DSL file, can take several seconds. For example, the latest Autosar metamodel (http://autosar.org) with more than 800 classes, takes around 5 seconds using Ruby 1.9.2 on my machine.

If instead we load the metamodel as a ECore model from a Ruby dump file and then transform it into Ruby classes using ECoreToRuby, the overall load time is down at a fraction of the time. In the Autosar example mentioned above, the load time is 0.2 seconds (factor 25 faster).

It’s important to note that ECoreToRuby creates most of the metamodel facilities on demand. This dramatically improves the inital load time, but since most of the time you won’t use all the metaclasses in a particular model, also the effective total load time will be much shorter. Fast metamodel loading is especially important if you run your tools many times, e.g. in batch processing.

The second big new feature in RGen 0.6.0 is support for models which are spread over several files. RGen now has a class called FragmentedModel which is used to represent the overall models. A model contains objects of class ModelFragment which represent the model parts, each associated with a file or any other storage location.

The main benefit of fragmented models is that fragments can be cached and reloaded individually. This way, model load times can be reduced drastically. How much depends on the performance of the non-cache loader. In case of our Autosar example, loading around 1500 XML files (about 60MB) takes 75 seconds. Loading the same from cache takes only 7 seconds (factor 10 faster). Caching on the granularity of model fragments ensures that most of the speed benefit is still there when some fragments (files) are changed.

Each fragment knows its model elements, can resolve references between them and keeps track of unresolved external references. Such a partly resolved model fragment is well prepared for being written to a binary dump file using Ruby’s fast Marshal#dump method. RGen’s DumpFileCache is responsible for loading and storing from/to the cache. Another new class called FileCacheMap takes care of the cache files in the filesystem. It protects them with a checksum and invalidates the cache in case the source file or the cache itself is modified.

The overall FragmentedModel can join model fragments in a very efficient way by only resolving the cross fragment references. It can also disconnect single fragments and replace them with a different one. For longer running processes, a FileChangeDetector instance can spot file changes on disk without recalculating all checksums.

Fragmented model support is especially useful in combination with RText. I will introduce RText in one of my next blog posts.

For a complete list of changes, please refer to RGen’s CHANGELOG.

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>