RSS Feed

Managing Android platform source with RTC

Posted on

My previous post on populating and testing a large Jazz repository could be tested with JMeter was primarily focused on the Work Item capability in RTC. I also needed to populate the repository with lots of SCM data and rather than take the “dumb” (ie. random) path as I had done with the Work Items, I wanted to use a more “realistic” set of artifacts.

I didn’t have to look very far: the Android platform which has apparently become the leading smart phone platform ( and 300+ million Android-based smartphone activations ( There are already a bunch of posts on Jim’s blog and on how to use RTC for Android app development, so I turned my attention to the Android Platform source, from the AOSP, which proves to be a different kettle of fish when it comes to SCM.

I went straight to Downloading the Source  and got to work. The source isn’t particularly huge (~6Gb) but does have a large number of folders (~250000) and files (~210000), which makes it well suited for my purposes.

While going through the exercise of sharing the source into RTC, I was somewhat diverted from my original purpose and this post is more about getting the Android source into and out of RTC than about looking at performance and response times.

Part 1: Componentize and share

The first thing to do before sharing the source into RTC is to organize it into components. To keep it simple I decided to componentize the  Android source as follows:

1.Each top-level directory (abi, bootable etc) except for “external” becomes an RTC component
2. Each of the directories under “external” becomes an RTC component
3. Anything else (like the Makefile at the very top-level) goes into separate component.

What this does is allow me to Accept, Baseline and Deliver changes to the components individually. For example, if there is an update to “apache-http“, I could choose to accept the updates, while another team/member decides not to. RTC is flexible enough to also allow accepting and delivering at finer-grained (change set) levels. Which means I could also just lump everything into one big “Android” component but that goes against good SCM practice. At the opposite end of the spectrum, I could also dive deeper into the hierarchy and break it up into even smaller chunks (components). Similarly I can add custom components, outside of what’s in the AOSP source, to hold stuff that my company is providing. The bottom line is that there isn’t one rule to fit them all when it comes to componentization, and RTC can handle whichever way you choose to go.

I already have a stream and repository workspace ready to go and here’s the sequence of gestures I use with the Eclipse client (running on Red Hat Linux) to share the source to my big RTC server, using the “libnativehelper” folder as an example. (Click the first image below for a slide show).

  1. Use File-> New Project to create a Project
  2. Browse to and select the folder to be shared
  3. Share the project choosing “Jazz Source Control
  4. Select the workspace and component to share into. I create a new component in the wizard.
  5. The top-level “.project” does not need to be version controlled, unless the directory chosen was already an Eclipse project. I also remove “bin” from the default ignore list, as there are “bin” directories that are part of the platform source.
  6. Once the Share completes the new component and change set is shown as an outgoing change in the Pending Changes view.
  7. When ready Deliver the changes.

Repeat the process for each directory that you wish to componentize.

Note that the  Share action can only be invoked at the (top) project level and not at the file or folder level. So as a special case I move the Makefile at the top-level of the Android source tree into a directory called “android“, load that directory as a project and then share the directory. Doesn’t this break the Android build mechanism? No, as we’ll see later RTC allows us to “cherry pick” artifacts from components and decide where they go on disk.

Keeping these points in mind, if I wish to use the RTC SCM CLI to share the source, I first must move each directory I want to share as a component to the top-level of the sandbox. I also add a “.jazzignore” file with

core.ignore= \

in it to top level non-Eclipse project directories. Once that’s done, the equivalent RTC SCM CLI commands to componentize and share are shown below.

lscm create component -r myrepoalias libnativehelper mycliworkspace
lscm share -r myrepoalias mycliworkspace libnativehelper libnativehelper
lscm status
lscm deliver -r myrepoalias

Part II: Loading or re-loading the shared source

Once the source is in RTC, split into components as above,  how do I get my mate Bob (and 500 of his mates)  to load all of this onto his sandbox, the way the original Android source was without having to figure out which RTC component holds which directory? Getting it out the way it was” is critical .

Hello “load rules“.

Once I’ve endured a little pain to componentize the way I’d like it, I use the RTC  “Generate Load Rules” feature to spit out a “mapping file” as it were, that captures which components (or parts thereof) go where on the disk.
I get to this feature in the Team-> Jazz Source Control -> Sandboxes node of the Eclipse/RTC Preferences.

Sandbox preferences

I click “Generate Load Rules…” , choose what I want the file called, where it should be saved (preferably a location that Bob and his mates can get to), not to create Eclipse projects and whether to use item IDs (better) or names, and click “Generate“.

Generate Load Rules

RTC generates an XML file that looks like this (abbreviated here):

<component itemId=”_pcmJ4cvqEeGrBKGZjCogOw”/>
<!– <component name=”my.libnativehelper” /> –>
<parentFolder itemId=”_pcvT0svqEeGrBKGZjCogOw” itemType=”folder”/>
<!– <parentFolder repositoryPath=”/” /> –>

<component itemId=”_45WWIsvGEeGrBKGZjCogOw”/>
<!– <component name=”” /> –>
<parentFolder itemId=”_45WWJsvGEeGrBKGZjCogOw” itemType=”folder”/>
<!– <parentFolder repositoryPath=”/” /> –>
<sandboxRelativePath pathPrefix=”/external”/>

<component itemId=”_OmzBYcvEEeGrBKGZjCogOw”/>
<!– <component name=”” /> –>
<item itemId=”_OvviAMvEEeGwtPxnFFxnOQ” itemType=”file”/>
<!– <item repositoryPath=”/android/Makefile” /> –>

I let Bob know and he creates his workspace specifying this load rule file and he gets everything loaded the way it was.

The load rule file can also be generated from the SCM CLI using:

lscm create loadrules  -r myrepoalias /software

Recalling the point about the SCM CLI requiring that directories to be shared should be located at the top-level of the sandbox, it will be necessary to unload and reload directories into their correct (original) locations, using the “-t” option to the load command,before running create loadrules. For example, since I would have had to move the “libnativehelper” directory to the sandbox root from the “external” folder, I need to first unload it :

lscm workspace unload -r repoalias -D -C my.libnativehelper

and then reload it into the external directory:

lscm load -r repoalias -t external mycliworkspace my.libnativehelper

All well.

However, the astute reader might have noticed that since I had moved it into a separate “android” directory before sharing it into the component, I would have had to carry out the same process (unload and reload) for the top-level Makefile, before invoking “Generate load rules”. Again this is simply done in the Eclipse client.

Unload the component, deleting the content.

Then Right-click the component -> Show Repository Files

01 Show Repository Files

Select and right-click the or file -> Load As

and select where the file should go.

03 Select location to load at

Part III The End

So I hope I’ve been able to show that it’s fairly simple to get the Android platform source into and out of RTC. When the number of components grows (I ended up with 150+ components) it does tend to be a little tedious. But there is a definite method to the madness and it’s easy enough to script/automate. More importantly the end result is well worth the one-time effort one person needs to put in. This is especially true where  of all of RTC’s other capabilities can be put to good use, and I will try to expand on those capabilities in the context of Android platform development in future posts.

5 responses »

  1. Dong Young Kim

    Excellent work on Android platform SCM.

  2. Pingback: Out with the old… | Freddy's blog

  3. Pingback: UrbanCode Deploy: Making deployment automation easy | Freddy's Jazz and DevOps blog

  4. What is the best way to manage Android platform source in case one is a contributor?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 45 other followers

%d bloggers like this: