Everything and the Mobile Software Universe…

Perhaps you don’t know, I’m the Tech Lead of the ELIPS Studio Product (feel free to try it for free here!) . For those of you who have already jumped on board, I really hope you enjoyed the first releases we have posted …and you have as much fun playing with it as we have building it!
We are pretty busy right now bringing some exciting stuff to the table, like, well, as promised the iPhone.
I just wanted to share with you the approach we have selected, and the path we are following.

iPhone : Widgets and the runtime part
We have decided to follow a different path than for the other platforms regarding widget handling.
As you all may know the S60 and WinMob widget toolkits are well, pretty dated and replacing it by some modern flex widget is a definitive plus. But in the iPhone case, this is the opposite, the iPhone has a very comprehensive, modern and user friendly widget toolkit, with physics, hardware acceleration, etc…and Apple requires you to use it to pass its dreaded Appstore approval.
So we had two choices here:

1. Redeveloping, in Flex the iPhone widgets so applications will  “as much as possible” look like real iPhone ones, or ..
2. Mapping the Flex UIComponent widget tree directly to real iPhone widgets to leverage the platform strength

The first solution has its advantages, like code reuse accross platforms, consistency when developing in the simulator vs a real phone opposed to the second one which is requiring a much bigger porting effort, impossibility to precisely simulate the application, BUT when you do an application at the end, it really feels like a real Objective-C one, following iPhone standards … but coded in AS3.
So yes we’ve taken the hardest path: we have followed the native widget route … and we are pretty happy about that
The biggest issue with such an approach is for us, not for you: this is the porting time.  We have to map all the iPhone widgets with their yet to be created MXML counter part and implement those new MXML widgets in a generic form (read pure AS/MXML) for the other platforms.
Today the iphone list is mapped, meaning that you simply have to declare a list with your own item renderer, data provider everything you know and love … and it will look and behave like a real iPhone list, with its unmatchable physics and hardware acceleration for smooth scrolling. Look at the following video for a very simple list with an item renderer in Flex and a data provider …no iPhone code at all but a real iPhone list anyway

Text entry (in place and not in place) is ported too.
Of course all the “normal”, non mapped widgets are working well and can be completely and transparently mixed with the native mapped one: you won’t notice the difference.
And we are at  that point: Shall we release it to let people try it even though a lot of widgets are not mapped? Or shall we work a little bit more to map all  iPhone widgets?

iPhone : the toolchain
As you have noticed, for now, our tools are Windows only (just to let you know I’m writing this article on my Mac ). Perhaps as you know too from our documentation we are compiling the MXML/AS to C++ for the applications, but also to develop our runtime and libraries: for the iPhone we have simply used our compiler and tools to generate a bunch of C++ files from our Flex SDK, alongside an XCode project then we transfer it to a Mac and we are able to build a fully native iPhone application, right from XCode, running in simulation and on device with C++ debugging….sweet.
In that case, for the application developer,  the flow is:

1. Develop in FlexBuilder With ELIPS Studio plugin on a PC (or better on a Mac, in a Windows VM, like Parallels, my favorite, or VMWare)
2. Generate the C++ files from your MXML/AS with the ELIPS Studio build button
3. Put this source tree on a Mac (if you were working on a VM … simply put it in a shared folder of your VM)
4. Open it in XCode, build, simulate, debug, deploy on the Mac

Of course in parallel we are working on a full Windows only toolchain, based on one of the patented unique OpenPlug technology: platform independent software components (used today in our WinMobile and Symbian platform and on Android also, oups, yes I’ve said that too look at the video , exact same list code as in the iPhone video, but in “Flex Only” mode, no natif mapping, pure MXML/AS for the list implementation:

Look at the performance on a G1 ).

Bottom line, we are wondering if it makes sense to release the current hybrid toolchain, or in the contrary wait for the integrated one.

iPhone : the next steps … tell us what you think!
So here we are today:  we are able to release an iPhone technology preview, with a limited set of native widgets, and an hybrid toolchain.

Beside this, the iPhone roadmap is pretty clear (and we are already progressing on it):

• Complete the UIComponents mapping to have all the iPhone widgets available
• Complete the Windows only iPhone toolchain
• …Port our tools under Mac, at least for iPhone development

I hope those insights helped you understand how our iPhone support will be completed, and I really would like to have your point of view and comments (either via twitter, in the forum or in this blog post):

• Are you willing to try our first shot for iPhone dev, with its limitations (very few native widgets)?
• Are you ready to develop in such an hybrid environment?

We need your feedback!

Found at All About Symbian:

Red Five Labs has just announced that their Net60 product, which enables .NET applications from the Windows world to run unchanged under S60, is now available for beta testing.

.NET on S60 3rd Edition now a reality?

This is really interesting: even the battle for languages/execution environment is not settled.

For years Mobility coding was tightly coupled with assembly code, then C and in lesser extent C++. The processor of choice is the ARM family (ok some others exist, but no more in the phone industry)…this was before Java.
Basically Java (the language) is no more than a virtual processor with its own instruction set, and this virtual processor, also called a Virtual Machine, or JVM in the case of java, simply do what every processor do: it processes some assembly code describing the low level actions to be performed by the processor to execute a given program/application.

On the PC other execution engines have been developed: the first obvious one, the native one is the venerable x86 instruction set: thanks to it all the PC applications are “binary compatible”. Then Java also, and more recently … the Macromedia/Flash runtime (yes Flash is compiled in a Byte Code which defines its own instruction set). An other big contender is the .NET runtime…with you guessed what, its own instruction set.

At the end it is easy to categorized the executions engines:

• The “native” ones: the hardware executes directly the actions described in a program, compiled from source code to a machine dependent format. A native ARM application running on a ARM processor is an example, or partially for a Java program that is running on an ARM with Jazelle (some Java byte code are directly implemented in hardware)
• The “virtual” ones: Java, .NET, JavaScript/Flash (or ActionScript, not so far from JavaScript: the two languages will be merged with the next version: ActionScript 3 == JavaScript 2 == ECMAScript 4) where the source code is compiled in a machine independent binary format (often called byte code)…But how an ARM emulator running on an x86 PC may be called? you guessed, virtual.

So why bother with virtual execution engines?
Java has been built with the premise of the now famous (and defunct ) write once run everywhere, because at that time (and I really don’t know why) people were thinking that it was enough to reduce the “cross platform development issue” to the low level binary compatibility, simply allowing the code to be executed. And we know now it is not enough!

Once the binary issue was fixed, the really big next one were APIs (and to be complete the programming model) … and the nightmare begins. When we say Java we only name the Language, but not the available services, same for JavaScript, C# or ActionScript. So development platforms started to emerge CDLC J2ME .NET framework, Flash, Adobe Flex, Silverlight, Javascript+Ajax, Yahoo widgets … but after all what are GNOME, KDE, Windows, MacOS, S60, WinMob ?…yes development platforms.

The Open Source community has quickly demonstrated that binary compatibility was not that important for portability: once you have the C/C++ source code and the needed libraries plus a way to link everything, you can simply recompile for ARM/x86 or any other platform. Hum I’ve made a big assumption here: you have “a way to link everything”. And this is really a big assumption: on many platforms you don’t have any dynamic link, nor library repository or dynamic service discovery…so how to expose cleanly your beloved APIs?
This is why OSGI has been made, like COM, Corba, some .NET mechanisms, etc : all is around component based programming, encapsulating a code around what it offers (an API, some resources) and what it uses (API and resources).

Basically an execution engine has to:

• Allow Binary Compatibility: Abstracting the raw hardware, ie the processor, either using a virtual machine or a clean build environment
• Allow clean binary packaging
• Allow easy use and exposition of services/APIs

But it seems impossible for virtual engines to dissociate the language(s) and the engine: Java …well for Java, ActionScript for Flash, all the # languages for .NET. An execution engine is nothing without the associated build chain and development chain around the supported languages.
In fact this is key as all those modern languages have a strong common point: developers do not have to bother with memory handling, and as all the C/C++ coders will tell you it means around 80% less bugs, so a BIG productivity boost, but also and it is something a tier one OEM told us: it is way more easy to train and find “low cost” coders for those high level languages compared to C/C++ experts!… another development cost gain.

A virtual execution engine basically brings productivity gain and lower development cost thanks to modern languages ….. we are far far away from “write once run everywhere”

But as we have seen before it is really not enough and here comes the real development environments based on virtual execution engines :

• .NET framework platform : an .NET VM at hearth, with a big big set of APIs, this is what I would like to know what are the APIs exposed in this s60 .NET port
• Silverlight : also a .NET VM at hearth + some APIs and a nice UI framework
• J2ME: a JVM + JSR + …well different APIs for each platform
• J2SE: a JVM + a lot of APIs
• J2EE: a JVM + “server side” frameworks
• Flex : Adobe Action Script Tamarin VM + Flex APIs
• Google Android: Java VM + Google APIs,… but more interestingly also C++: as android use Interface IDL description C++/Java interworking will work (I have some posts to do about it)
• …and the list goes on

So at the end what really matters is the development environment as a whole, not simply a language (for me this is where android may be interesting). For example the Mono project (that aims to bring .NET execution with Linux) was of few interest before they ported the Windows Forms (Big set of APIs to make graphical stuff in .NET framework) and made them available in their .NET execution engine.

What I haven’t mentioned is that the development costs gain allowed by modern languages comes at a cost: Performance.
Even if Java/.NET/ActionScript JIT helped partially for CPU (Just in Time compilers: VM technology that translates virtual byte code to real machine code before execution), it is still not the case for the RAM used, and in the embedded world the Moore law doesn’t help you, it only helps to reduce silicon die size, to reduce chipset cost, so using a virtual engine actually will force you to … upsize your hardware, increasing the BOM of your phone.
And it isn’t a vague assumption: when your phone has to be produced in the 10 millions units range, using 2MB of RAM, 4MB of flash and an ARM7 based chipset helps a lot let you make money selling at low cost….we’ve spent some nights/days optimizing stuff to make it happen smoothly very recently

At OpenPlug we have built a nice execution engine, not virtual, running “native code” on ARM and x86, with an easy to use service discovery mechanism: a component platform for low cost phones. Then we have added a development environment with tools and middle to high services….we have learned big time and sometime with pain all the bricks needed for such an environment! …

A key value may be around one framework and multiple execution engine for easy adaptation with legacy software and productivity boost for certain projects/hardware, or some parts of the software.

Because once you know how to cleanly make some code running independently from the hardware, you have to offer a programming model! Implying how to share resources between your modularized pieces of code…and in that respect execution engines are of no help, you need an application framework (like OpenPlug ELIPS, or Hiker from access, Android is about that, but also S60 and Windows Mobile): it will abstract the notion of resources for your code : Screen, Keypad, Network, CPU, memory, … this is another story, for another post!

Please comment!
Thomas

Here is a little Open-Plug video that shows only the “step 0″ of our technology, here are some remarks:

• This is a real-time one, so it may be long at moment…especially when loading a new system (this is the case after the red and blue UI).
• The first UI (the red and blue one) HAS NOTHING in common with the second gray and red one called OPUS, really, only the porting layer and our application framework are left on the board, and on the SD card you have all the components that are implementing the UI and its services.
• As you can see it is an NXP board (platform: 5210), we do exactly the same on other platforms also (TI locosto and Neptune for the publics ones )…and on all the platforms, the code is the same, developed on our PC simulator, and it is C/C++ not java of course!
• I say “step 0″ because this demo is basically showing a complete MMI replacement (the O-phone demo), but our offer is in fact completely dynamically modular, as shown briefly with the sudoku download
• Yes Doom is there up and running : intern has ported the SDL library …and so Doom was in the package, sweet to have Open-Source code on such a platform in a matter of days!
• You can see some background play features: MP3 is in the background you edit a message and an icoming call is poping up: applications and services doesn’t know each other, the multimedia player doesn’t know anything about the call, nor the messager. The sound of the MP3 is autmatically prehempted for the call, but not by an ad-hoc way like in many high level OSes, you can see it as a window manager for the sound. Our focus is really on integration and modularity:
  • For static code with our component based technology
  • For services integration with a really original and system wide resource management system.
• We like Placebo

So have a good show

During the last LiPS forum face to face meeting we had in Seoul (thanks again to Mizi Research, great organization), Access/Palmsource presented its Application framework, now Open-Sourced: Hiker, and one point has raised a little debat: it is fixing the way applications can be layered, interacting with each others (One running Application with the User Focus at a time, no multiple instance of the same application): so it is hard to implement some Vodaphone Live! use cases requring multiple launches of a same application like : You are in your Address Book, you recieve an sms, you answer, you look for a contact launching again your address book from the sms, the address book will be on top when you close it you are back to the sms and when you close the sms…you have your first address book below with the same state before the sms reception, at the beginning.
It’s not a secret, I’m a big old time palm fan, but now I’ve a better understanding of what is a phone and the industry around it: Hiker seems great and may bring good usability practices, etc BUT it will be difficult to fullfill some Operators or OEM requirements, cause it fixes the way the User Interactions have to be done: it reduces innovation (good discussion about that at MobHappy)
Apple has made an Apple phone from A to Z, from Software to Hardware (see this Michael Mace article if you were on another planet, and this nice round-up at TomSoft for the 58# Carnival)…but if you (as a phone manufacturer) want to make a phone you designed, chance you are an hardware company, and you make money with it:

• If your are an ODM, you have an OEM specification.
• If you are an OEM you have your own specifications, matured for years.
• If your are an OEM or ODM you may also have some Operators strong requirements or sometimes complete product specifications (Vodaphone Live!, NTT Docomo, or some iMode recomendation from Bouygues for example).

So:

• Either you will go shopping for an external software solution (like Moto with TTPcom and many ODM and some other big players)
• Or waste millions of dollars in an unmaintenable solution (I wont’t throw names here … even if the list is long). Because it is not your job.

If this external solution locks you in a single type of product, so in a niche…you are screwed, plus the royalties are high and the hardware minimums are stratospheric compared to the real phone market.

Choosing Access/Symbian/WinCE : Choosing the OS equal make a fixed and finished product:

• impossible to comply with operator specs
• same for OEM
• : it is why nokia has made s60 and SE bought UIQ (I often take as an example the Text Input specifications: it accounts for around 25% of a complete Phone UI specification….it is very hard to change it in high level OSes…but it is a big brand asset for many OEMs)

Your are not making your phone but a windows/palm one

Interestingly solutions coming from the US are more monolithic and impose a user experience (palm, winCE, sky mobile media has bought eSim an israeli company to have a kind of UI customization), compared to EU ones (symbian is nothing without a customization, Open- Plug, Digital Airways and TAT for UI, etc are all about customization, Qtopia in less extent)…there is certainly something cultural here (many time the “not invented here” prevails in the US leading to more integrated products but moonolithic, while EU is by nature more diverse and consensual…just a guess here). Somewhat related: good read at Vision Mobile about Mobile Linux.
At the end I’m not sure an Operator is really able to specify a full User Experience, focusing on usability (no no , it is not sarcastic ). Software companies, can do it in some extent, but at the end only OEM with strong brand assets, big worldwide tracking records of mature products for all cultures may know those little things that make the difference.

For sure new comers with good ideas (like Apple here) and a good knowledge of software AND hardware can do great stuff for a niche market…but selling to China, Arabic countries or India is another beast, it is a 2.7 billions users game!

Perhaps you’ve heard a lot about ULC (Ultra Low Cost) cellphones in the industry (many companies are aiming this market, see The Motorola F3)…but they are targetted to new markets, where text rendering/input is really more complex than our relatively simple alphabet: it is mandatory if you want to reach the mass of Indian users, Sri Lanka, Cambodge, Hebrew, Arabic, Philippina , Chineese…and you want cause “texting (smsing)” is a big thing: else your new emerging market will be limited to eastern Europe or South America, and be sure that text messaging will continue growing bigger. Look at the buzz around hotxt (read this mobHappy article about this new service, even if it is all but positive about it or at SMS Text News, please check the comments on this one… ) or Berggi (Berggi GigaOM article, I’m not sure about the business model of such an offer, 10 bucks a month to send sms!!). IM is perhaps the next text heavyweight champion (check this techCrunch article about the next bif ideas around IM)
So Text IS big and will stay big…so emerging markets have to have the tools to leverage this potential!

Two different issues to be fixed araise:

• Text Rendering
• Text Edition

In fact text edition is not so diverse and different accross the languages: the T9 you know is not so far from Chineese input. At the end you only have two kinds of predicitve text input methods:

• One for alphabetic languages, the one you have on your phone today. For those languages multitap is also of course heavily used
• One for “ideogram” languages (hum, only in China/Taiwan and Japan (kanji) in fact ), with Pinyin, Strokes, Bopomopho input being some variations

And after having implemented both in real life here at Open-Plug…the alphabetical one is in fact more complicated!
Two companies have the lion share for the predictive engines:

• Tegic (now AOL) and its T9 offering, more Western countries oriented
• Zi corp (eZiText, eZiTap), more Eastern oriented

Both have great products and large and complete languages databases, for me their biggest IP. Cause once you have a predicitve engine the real pain only begin…the UI and user interaction around those dictionnaries is really really complex:to give you an idea, the specification of text entry methods and use case represents more than 25% of the whole UI specification! and of course, for each phone/vendor/carrier it changes, so integrating T9 or EZiText really means if fact tayloring YOUR existing text entry framework for a given specification, the predicitve engine being only a little part of the whole thing. We ended up, in collaboration with LiPS to create a real “Text Entry Framework”…

So let’s move to the text rendering part…
Many companies are developping solutions to render text…so why such a fuss? After all it is only about drawing characters on the screen no? For us, with our simplisitc alphabet european languages, it sounds like something trivial…IT IS NOT, cause complex languages are REALY COMPLEX.

• First you have the magic around Bidirectional text (for those crazy men that are writing from right to left, Hebrew, Arabic…I’m sure they say the same for us writing from left to right )… you will say it is only printing in reverse….you couldn’t be more wrong, simply look here for the full specification of the bidirectional algorithm, hum, simple isn’t it? How to insert left to right text in a right to left paragraph, how to wrap in that case?
  
• Then you enter the brave new world of GSUB, GPOS ans co. In some languages some characters are mixed together and inversed to form a new character: the logical ordering (in memory) is really far from the display ordering. Champions in those kind of games are Devanagari, Indic, Thai…
  

So to handle all those languages, a solid text layout framework (for character clustering, reordering, wrapping) is needed, and a good font renderer. The open source community has developped Pango for the layout and Cairo for the rendering. ICU, from IBM is also a well known open source text layout engine.
Anyway those projects are huge, a big linguisitc knowledge is needed (try to debug a devanagari string…) so to have a quick working solution the industry offers you partners…and many exist, showing that this is a real issue!

• nCore :a little finish company that has a full Layout+rendering engine and a rich text editor, we are working with them already : their solution is really great, fast, light, etc, plus they are darn good at what they do. Their core busines is not font, but rendering.
• Tegic (T9) has a solution (seems to be based on nCore technology), adding their own features and a good integration with their text input solutions
• ZiCorp has also a solution that can be integrated with predictive text input solution
• Monotype, a 100 years company that invented the Arial and Times New Roman fonts…they know well how to render texts , their market is more toward the fonts themselves
• Arphic a taiwaneese text rendering vendor, I don’t know much about them
• Bitstream, a US text rendering vendor, with a solid technology. Also more “Font oriented”, and vector based
• Microimage, I’ve heard of them thanks to 3GSM Asia, but for now I don’t know much about their technology

So to play seriously in the ULC field the phone software needs a very solid text solution…and really it doesn’t come overnight! And be careful when designing/implementing/choosing your solution/partner: it won’t run on smartphone but on ULC phones, so no extra RAM nor CPU, and here CPU IS a challenge. Aslo don’t pass on it, people are really not willing to simply learn english

Check this annoucement: Texas Instruments Application Suite Ecosystem Boosts Development of Affordable Multimedia Feature Phones: Financial News – Yahoo! Finance… Just to let you know that things are moving !

[...]The process of porting, validating and integrating software takes significant monetary and time investment. TI is committed to helping customers maximize those investments by offering them a choice of pre- integrated application suites[...] TI customers can select an application suite such as Motorola Ajar, OpenPlug’s ELIPS, Sasken’s ARIA and SKY MobileMedia’s SKY-MAP(TM)[...]

Texas Instruments Application Suite Ecosystem Boosts Development of Affordable Multimedia Feature Phones: Financial News – Yahoo! Finance

Just a little post to share this good news :GSM Association Press Release 2006 – Microimage & Open-Plug Take Top Prizes at Inaugural Asian Mobile Innovation Awards…great!

In this blog we talk a lot about developpement frameworks in the mobile space, which are close to “traditional” framework (like GNOME, Qt, Windows, MacOS, even Java in less extent), but newcomers are popping … from the web space.
I’ve played recently with a great CMS (hum : a blog engine on steroid) Drupal to developp as quickly as possible an in-house test cases/release quality report system for our test teams, a weekly tracking tool for my team and some other stuffs: I’m NOT a php/javascript/XML/HTML junky, I’m a C/C++/algorithmics dev, and … doing that was EASY and FAST, and I must say efficient.
The biggest positive points that have popped out:

• Documentation of PHP/Drupal : amazing, clear, dense, …neat
• Data formalization (RSS feeds and XML)
• Powerfull presentation framework that let me concentrate on the job

If we compare those point to exisitng frameworks, some are really missing : Documentation of course is not only many time missing, but when it is present, it has been written by code monkeys, certainly good at coding and architecture, but not for teaching and giving fun (I’ll quote here one of my Kathy Sierra favorite post never underestimate the power of fun). Data formalization: it is a joke today in the C/C++ world, where is easy serialization? Database?. Presentation framework : ok you have some exisitng but you still have to fight with widget mechanics…boring and bug prone.

It is where new web frameworks may bring something new, I’ve spotted 2 at this time:

• Ruby on Rails
• OpenLaszlo (found again via TomSoft)

Ruby on Rails is really pushing the MVC (Model View Controller see my post here and here) paradigm to its limits, with a clean high level language….I’ll try to use it as soon as possible.

OpenLaszlo claims to bring the desktop power to web applications … and the approach seems greats (look at the examples) : the UI is described in clean XML, then everything is compiled in Flash or DHTML, AJAX stuff/javascript are hidden.

So it will be time to bring this kind of technologies to mobile developement … and constraints… Flash today is perhaps too RAM and CPU consuming (but I’m sure it will change). Making things too easy has also its drawbacks, especially in an area where customization is a key differenciator, see also this Kathy’s post about ease of use.

At Open-Plug we already have a clean, all recoded GTK that is running on Ultra Low Cost phones with no issues, for me the next step would be to add some more formal and high level UI descriptions, and some better data modeling to come close…We have begun, and it is exciting!

A little post to share this news with you :alarm:clock euro: OpenPlug Raises €15M And Other Fresh Euro Deals Archives
(Perhaps) As you know we are pushing really hard to bring something new to mobile phone software developpement and integration … this will definitively allow us to go even further! … a little of self satifaction is sometime a good thing after some pizzas/release nights .
Update : Fred Destin talks about us also…
Update 2: A mixed comment about OpenPlug Strategy good read even if, of course I disagree on some points:-)

GSM Association Press Release 2006 – GSMA announces shortlist for the first Asia Mobile Innovation Awards We are really proud to be part of this shortlist!