First Website Running “on The Ball on Azure” Goes Live!

Thanks to hard work put in by the team consisting Citrus, Caloom and Aalto Global Impact people, we can finally present the first website running on top of The Ball – that is running on top of The Windows Azure.

The website in question is the site of Aalto Global Impact (www.aaltoglobalimpact.org), the partner with whom we designed and built the “Open Innovation Platform” – the first concrete instance of “The Ball”, that was released for general developer availability few weeks back.

The website is maintained with quite basic CMS back-end functionality. Part of it’s html is served as static content, that is background rendered (the single articles), part of it is dynamically browser-template rendered (the isotope/masonry tiles). This design is partially based on earlier generation html rendering support, but it serves its basis for making the request sizes of single articles to be just the display html size.

The details of the logical design and technical architecture is mostly covered in the MS TechDays Developer Availability Launch Post, but I find following details worth while to mention:

• Website is served directly from Azure blob storage; making it extremely scalable
• It’s running currently on top of 4 x extra small web roles and 2 x extra small workers (We’re experimenting whether its better to use small or extra small roles)
• The runtime environment is designed to serve and consolidate any number of websites
• The cost metrics (CPU, network, storage) can be measured accurately and costs can be assigned accurately per-site

The above factors mean, that the platform instance/runtime can consolidate any amount of websites in a controlled fashion. The “economy of scale” applies to this consolidation; single site traffic spikes make much less impact on the average.

In addition to traditional websites, the same applies to all “apps” and services described earlier. So in addition to cost-efficient consolidation, the upgrade path to large scale distributed system is a straightforward upgrade path of simply adding the functionality on top of the traditional website.

If the environment is controlled by the hosting party – and the platform is audited and controlled, the consolidation can be spread among various environments and various application providers, while the privacy guarantee holds.

Some of the things I listed here are still under construction, but the simple distributed architecture makes them relatively straightforward to design and implement. And in a manner, that competent architects have no problem to understand.

This brief announcement post became bit longer than I aimed it to be For more details about the platform, the previos developer launch post serves better summary of what to look and where.

Cheers,

Kalle & The Ball / OIP team!

“The Ball” Launched @ Microsoft Finnish TechDays!

Our long focused efforts were rewarded, when we could finally launch “The Ball” and “Service Operating System” to general developer availability. I was really happy to be able to do it in the annual Microsoft main developer event, TechDays, that was held in Finlandia Hall this year on 5th and 6th of March.

While we’ve built the actual platform for more than a year (starting from concept level prototypes in early 2012), bringing and polishing the “Hello World” start from scratch demo made me really realize how far we’ve reached. We have raced “against” much bigger players on the digital lab field (even in Finland), trying to persuade them to join us to build this completely open (WTFPL licensed) platform with us. Given my background for working on early stage O/R Mapper (Nolics.net) on .NET, working with Visual Studio extensions/add-ins to really reach down to as-good-as-possible (given the resources) end-developer experience, I’m quite proud how streamlined the “get-develop-deploy” cycle for “The Ball” already is.

The presentation given now was rated as 400+ (expert architects), because the competent architect(s) still get the best out of its current shape – and are the ones who can take the current release and develop it further to suit their particular needs and purposes. However the demos are at the level of getting to run “Hello World”. And more than that, those demos showcase a way to start very standard HTML/jQuery UI development on top of the distributed back-end.

The development of “The Ball” has been completely open from the beginning (some areas of the codebase could use heavy cleaning even still), but the amount of “ADM Powered” code is so small, that it justifies even complete rewrite for the parts that don’t suffice for the purpose. Bottom line is, that now we finally could reach down to the level that any developer can get, build, and run the platform.

So without any longer stories, here’s what we presented and launched:

• GitHub Repository for the complete platform and HelloWorld example.
• The presentation slides The Ball / SOS on Azure sessions separated: TheBall @ TechDays Part 1 (PDF) and TheBall @ TechDays Part 2 (PDF).
• New videos added under “The Ball – The Information Ecosystem” in Demo Videos section. (I will add architectural deep dive videos later on)

For additional reading, in case the ADM and The Ball context is not familiar I’m linking the following:

• The Ball videos of Aalto / OIP platform, from August 2012 the abstract definition is completely valid now
• The workshop agenda paper “Workshop on ADM and The Ball“ describing ADM and “The Ball” paradigm. Initially used in the workshop that we held together with Free University of Bozen-Bolzano on October.
• Whitepaper about information valuation “towards citizens as digital innovators“
• For ADM best sources (including links to technology hands-on-demos) in addition to demo videos, I’d suggest our MS TechDays 2012 Material.

The development tooling requirements are as following for the Azure platform:

• Visual Studio 2012 (should also work with VS 2010 SP1)
• GitExtensions (for submodules) - the VS2012 Git support doesn’t suffice
• Azure SDK 1.8 and Amazon Web Service SDK (for email services)
• T4 Toolbox and Tangible T4 editor (free) for good ADM/T4 experience.
• For ADM abstraction modeling Altova’s XML Spy is one of the best tools
• For code-level implementation ReSharper from Jetbrains has still our preference

We have initiated dialogue with various open-source solution providers (especially on Big Data space) as well as critical data hosting parties, who have clear interest to port “The Ball” to run in datacenter hostable platform. For any interest on those areas, we’d be happy to hear about you!

Warm (yeah right) and Sunny Spring from Helsinki, Finland

Kalle (+ the rest of The Ball Team!)

Sneak Peek to Service Operating System…

Formalizing the model for “Apps” in the information and service ecosystem has kept the team quite busy. In the recent discussions about “How is The Ball platform related to the services?” we came to quite a good explanation of the whole concept.

The Ball can be thought of Service Operating System. It is target platform agnostic, so it deploys on any technology stack with ease (thanks to ADM). It runs everywhere natively… so it’s logical platform rather than a platform. The design models however do require certain primitives:

• Data Storage/Retrieve (storage space & storage transactions)
• Data Processing (CPU time & runtime memory)
• Processing Queues (storage space & storage transactions)
• Transmit (transmitting the data, network bandwidth)

While the blog has been silent for way too long and we’re still polishing the whole concept I wanted to share few images, that might make sense to those already familiar with “The Ball” (for those NOT familiar, but interested, “The Ball” is described in previous handful of posts starting from August 2012).

Service Operating System (SOS)

“Apps” running in security context sandboxes. Cross integration and trust management between the providers, all within control of the user.

Service Operating System Integration Overview

This image is first one in the series describing the SOS overall and comparing it to traditional “siloed” digital service environment. In this model the integrations are controlled and driven by the user, service and “App” providers are to provide the model-compatible services and converters to enable plugging.

Master Information Management

Another topic that we’re going to better cover is information flows within the processes and implicit master information management that is supported by the core SOS engine.

Information rendering with Master Information Management

The master information flows are supported within security contexts and between separate contexts with active “push” type subscriptions and “pull” type monitoring.

I’ll cover both the areas with better concrete examples on the running platform soon(ish). The master information management is already in full-action on the current building of “Open Innovation/Collaboration Platform” and we’ll properly aim to demonstrate it with resource level monitoring and cost-based billing on “Apps” running on the platform.

Happy beginning of 2013,

Kalle & the rest of “The Ball” team!

First International Workshop About “The Ball” Held

Thanks to Pekka Abrahamsson and Daniel Graziotin from the Free University of Bozen-Bolzano for organizing the audience and technical bits of video-conference, the first international workshop about ADM and “The Ball” was held October 29th and 30th. This workshop was natural continuum of our technical seminar sessions of ADM in Microsoft’s Finnish TechDays (in March), business opportunity session with Finnish TechInd and Microsoft (in May) and the introduction about The Ball with Aalto Global Impact (in August).

The workshop was held in two separate two hour sessions and focused on higher level overview with technical detailed demonstrations on key aspects for above platform reusability of the ADM-method.

Session 1 (2 hours)

Initial overview of the methodology.

• ADM Innovation Background

The original need to control the developers’ coding outcome, unified developer experience for swift entry to the team and it’s practices. Starting from Word-based guidance, ending to full-tool-supported self-maintained automation.

• Common Application/Solution Architecture Blocks

Focusing on network-service client-server proxy layer. Expanding the logical layer to be found on any application down to method calls within single application.

• Demo

Using existing abstraction – experience from developer perspective, explaining abstraction components and overview of creating/modifying one.

Session 2 (2 hours)

Explaining gradually how to get from local use to libraries and further on to open source ecosystem wide solution with existing infrastructure (XML, T4, Git based mainstream available tooling).

• Logical Operation Structure

Process/workflow like abstraction, with no engine, framework or even library requirement; thus immediately cross-platform available with simple code-orchestration. Going through the information flow from such sturcture to documentation and status tracking.

• Gradual Migration from Existing Software

Explaining the overview and steps to migrate existing software layer to higher level abstraction. After the migration the cross-platform generators become immediately applicable even for the legacy software in it’s maintenance state.

• Ecosystem/library usage

Explaining separation of generators from the higher-level abstraction content, so that the abstractions become reusable and independent of their actual using instances. Introduction to the solution that semantic signatures for service (and thus down to method) layer can be used to LEGO-like connect and tailor functionality down to end-user application/solution level.

• The Ball

Ran out of time to dive into The Ball. Referred to existing material and deferred the introduction for follow-ups on this initial workshop. The Ball is “just” a concrete take on information owner recognition combined with semantic information and logical operation control. It enables complete and pure logical, distributed information processing flows and honors the ownership and authorization boundaries on every step.

Future Workshops

Future workshops are to be held based on growing interest. The introduction and preparations for hands-on workshop is briefly described in “Workshop on ADM and The Ball” paper. The future more advanced workshops will be based on the experience and feedback from the current early adopters of the method.

If you’re interested about the results and/or participating on the next workshop, please contact us directly or follow up us in this blog or through LinkedIn group “The Ball – The Information Ecosystem”.

From rainy Helsinki,

Kalle

Information Fundamentals – To The Cloud and Beyond

After extremely busy August and September, we finally are reaching the level of finishing the core of information management platform alongside standard CMS (as in Content Management System) functionality on top of it.

During the final implementation on top of Windows Azure stack (basically dancing on top of the core fundamentals of store/retrieve information and queue processing) I feel so alive again. During the last two years of developing ADM I’ve purposely driven myself to explore the technology stack also outside the Microsoft’s (.NET) offering to bring the development ADM-automation in-line for also the other mainstream mobile stacks (for Android and iOS/iPhone). Now coming “back home” for the recent two months push to stretch all my .NET architect’s skills and built completely distributed solution on top of Azure’s core tech stack (blob storage, queues, background web rendering) – has revealed something truly beautiful !

During the 2012 we’ve concept level built a full information/digital service ecosystem core based on Git; introduced in the ecosystem overview few months back. However all of those steps were still something that required software development professionals to work on. Only during the development of “Open Innovation/Collaboration Platform” that we’ve worked with Aalto Global Impact, we’ve finally reached way simplified information management. The Ball model, that was publicly introduced in our early August seminar material got it’s final shape through the practical collaboration model design during August.

While the actual “deploy-yourself-to-Azure-or-on-premises-Microsoft-stack” demonstration is soon to follow (git repo already available), I won’t go into lengthy detail describing the models, but to give brief summary of the core design aspects.

1. Authentication and Authorization – separated

Authentication is OpenID/OAuth supported with trusted providers; authorization is based on “ownership” and validation of real-world email address.

The fact that you would trust in real world met person by his/her emails for business agreements (not arguing whether legally bound or not), gives basis for “The Ball” also use regularly validated email address as a token of authorization. We don’t trust your Google/Yahoo/Facebook/Live profile information at all; but for the tecnical authentication they do more than fine.

Collaborator AuthN and AuthZ in The Ball

2. Data Authorization Scopes – defined

We have designed a model where the information has three logical places to exist; public, group owned (membership authorization controlled) and account owned (private, personal). Public does not require any authentication (for humans or digital applications) to access; group requires collaborator role to modify, viewer role to read (same for OAuth token granted applications) and account is bound to individual’s private storage.

Every group or account can also choose to publish any information they want to. In the first demonstration this will be implicitly as CMS functionality; blog posts and web-pages, but of course also machine-processable form of pure semantically defined information.

Information Authorization Scopes

3. Digital sanboxed “App” platform – implicit

As basis for the information ecosystem defined in the “citizens as digital innovators” whitepaper is built for separating the user-controlled confidential content from the information models, the platform implicitly allows building “Apps” with existing information models – without the user/group ever having to trust and share his/her confidential data for the app developers (which is more or less the case with the current “siloed” cloud offerings). In “The Ball” the “Apps” are to be sandboxed to only access the information within the account/group containers.

The “Apps” don’t have to deal with any complex distributed software architectures, as there are none in the context of Apps themselves. They simply process information from one model to another (be they critical medical/patient records or as innocent as family photo albums). The platform’s core will take care of queuing the processing and (if user chooses to) also update the published content.

When user is in control of his/her own information; the information valuation (described in the whitepaper mentioned before) becomes real – way beyond today’s Facebook/Google+ “textual” information valuation.

For those parties with critical requirement that cannot run their instance of “The Ball” in the Windows Azure (or Amazon, or any other) cloud provider – the option to run it on-premises is as valid and comes with exactly same benefits as the cloud-based deployment. The Ball can still be cross-integrated with other instances; whether cloud-based, local based, or even mobile-device-based “micro” instances.

Where’s it at?

For developers familiar with Azure stack, the current development is happening on wide-open (note; some issues with git submodule config in the current repo – but they shouldn’t cause real issues):

https://github.com/kallex/CaloomUI

Edit: Was accidentially referring only to “CMS-UI” part. The core platform (unpolished for now) repository (that also uses the “CMS-UI” part as a submodule) is available at:

https://github.com/kallex/Caloom

However the current development is not UI-design neutralized for “clone-at-will”, but in need of such a version before we prepare one “clone-from-here”, please don’t hesitate to contact us.

We will relatively soon update with the neutral “clonable” version; including the mobile-client stacks for Android, Windows Phone and hopefully iPhone and MeeGo as well (not necessarily in that particular order).

I will also then write in detail how completely distributed the model is (as it might not become apparent by just mentioning that account/group/public processing are independently managed). At this stage suffice to say, that “The Ball” can scale to fill the whole Azure datacenter (processing) capacity, then reach to Amazon’s offerings… without any of the “Apps” or end-users seeing any difference in behavior. (Of course also safeguarding our business and solution not to be Microsoft, Amazon or any other clould provider bound).

Wishing chilly Autumn (with rains included – where applicable) from “The Ball” core team,

Kalle

The Ball – Session with Aalto Global Impact

Thanks to everyone who attended in our joint seminar with Aalto Global Impact about the business opportunities of The Ball and the Aalto’s new Open Innovation/Collaboration Platform!

The session slides about the introductions and the main content are available as following:

Aalto Global Impact Introduction

Caloom in The Ball

Aalto OIP – The Ball Business Opportunities

The video recorded afterwards from the OIP/The Ball content (split to three different videos) can be found in the Demo Videos section of the blog.

You can best find and track us nowadays in LinkedIn Open Groups:

• “The Ball – The Information Ecosystem” for the generic “The Ball” matters.
• “Inclusive Business” for Aalto’s Open Innovation/Collaboration Platform opportunities and matters.

Back from vacation and (hyper)active again

Kalle

P.S. The reason for not making the promised summary around midsummer was, that we were so busy starting the OIP development with Aalto in the early July. There wasn’t much to post about that yet, but little point to post about anything else either.

Finnish Technology Industry, Citrus and Microsoft Seminar

Thanks for everyone who attended to the “business side” world awakening seminar of software development automation – in context of distributed software development.

The seminar slides of the ADM-presentation held by me are available here (including the teaser material beyond the presentation) :

ADM – Teknologiateollisuus, Citrus, MS seminaari (PDF – In Finnish)

We are to follow more material on currently forming up ecosystem. Much to prepare before vacations, even more to come after

I’ll be entering one more post before vacations to summarize our last year’s progress. I try to be less destructive and more constructive this time. After all, we learned how big of a thing we can create with this; last year we only knew we’re going to take all the existing status quo down…

Cheers!

Kalle