13Mar/1324

Preview of the new Managed Client for Windows Azure Mobile Services

Note: This is my personal blog. Although I’m currently doing an internship at Microsoft and this post is about a product released by Microsoft I worked on, the opinions on this blog are my own. They do not represent the opinions of Microsoft in any way.

Update: Carlos Figueira has a detailed list of breaking changes at his blog.

Update 2: We published an update to fix the issues with Windows Phone 7.5. The update can be found here.

Today we are releasing a pre-release version of the new Managed Client for Windows Azure Mobile Services on NuGet. This release adds several new features and also adds support for Windows Phone 7.5.

The managed client now supports the following platforms:

  • Windows 8
  • Windows Phone 8
  • Windows Phone 7.5
  • Portable class libraries.

Windows Phone 7.5 support and Portable Class Libraries have been among the top requested features by our customers!

uservoice

Great! How do I get it?

The preview is available as a pre-release NuGet package. To use the package you need at least NuGet 2.1.

Note: This version is not officially supported. The latest officially supported client is available as a stable NuGet package or as an extensions sdk.

nuget-pre

To be able to see the package in Visual Studio you need NuGet 2.1 and you need to explicitly tell Visual Studio to show you pre-release packages.

includepre

New features

Beside support for new platforms and bug fixes, this release also adds a bunch of new features:

  • Enum support. Enum types are now supported in queries and for serialization. Enums will be serialized automatically into strings by the client. If for some reason you want to serialize enums as integers, you can still use the following technique to expose the enum as int for serialization.
[JsonProperty("value")]
public int ValueInt { get; set; }

[JsonIgnore]
public EnumType Value { get; set; }
  • Nullable types. Nullable types are now fully supported for queries and serialization. You can read more about Nullable Types here.
  • Contains support in queries. We already supported string.Contains in queries, but now you can also write Contains queries on lists. These queries will now automatically be translated into a sequence of “or” statements:
List names = new List() { "John", "Tom" };
from p in table
where names.Contains(p.Name)
select p
  • MobileServiceCollection. We added a new collection that replaces our previous MobileServiceCollectionView. This class had some problems and was less portable going forward. You can read more about the new collection further on.
  • HttpMessageHandlers. Service Filters available in our previous release are now replaced by HttpMessageHandlers. These classes are part of the recently released HttpClient library and are reusable by everything that uses HttpClient. More on this also further on.
  • Improved unit-testability. We removed the sealed keyword from most of the public facing classes and added interfaces for some of them. This heavily improves the unit testability of the library for us and make it easier for developers to substitute our classes for mocks.

Portable Class libraries

Currently we have separate clients for Windows and Windows Phone 8. This release unifies these platforms by moving towards a Portable Class Library. Portable Class Libraries offer several advantages for developers, since they can now share their codebases using Mobile Services more easily between the different platforms. Also it allows us to rollout changes faster and brings consistency across the supported platforms. You can read more about Portable Class Libraries on MSDN.

Our portable library targets:

  • Windows Store apps
  • Windows Phone 7.5 and higher
  • .Net 4.5
  • Silverlight 4 and higher

This does not mean we support all these platforms. The Portable Class Library need a platform extension assembly to work correctly and it won’t work without it. We only provide platform extension assemblies for Windows Phone 7.5, Windows Phone 8 and Windows Store apps. If you need to support other platforms, you can get the source code from GitHub. More on platform extension assemblies further on. We are considering adding .Net 4.5 support in the future.

Architecture

Although Portable Class Libraries are a great addition to the Managed Mobile Services client, we cannot move all of our code into the Portable Class Library. Some functionality is only available on one platform and other functionality differs between platforms. Therefore, to be able to use the new Managed Client you also need a platform specific assembly, which contains code to be able to use features available on that specific platform. UI authentication is an example of a feature that works different on different platforms and is not portable.

diagram

The design of the managed client is roughly as shown in the diagram. For you as a developer this means you need two assemblies for your Windows Store app to use Mobile Services. For Windows Phone this works a little different. For Windows Phone there is an additional assembly. This assembly contains functionality which is not allowed in background agents. In our case that means that login functionality involving UI lives there. So, if you install this package into a background agent project, make sure to remove the reference to Microsoft.WindowsAzure.Mobile.UI.dll.

With the platform extension assemblies we take an approach similar to what the Reactive Extensions team earlier did with Rx 2.0. We are constantly evaluating what platforms to support with our Portable Class Library with respect to the functionality we can actually put in the portable part, because support for more platforms in general means we have to move more functionality to the platform extension assemblies.

A new collection

MobileServiceCollection is our new collection type that replaces MobileServiceCollectionView going forward. The functionality it offers is similar to its predecessor and it offers more extensibility.

  • Support for paging. Simply give the collection a page size as part of the constructor and it will request the data in batches.
  • Client-side LINQ queries. We acknowledge that OData does not cover all query scenarios for users. To make this as easy as possible you can specify a custom selector function over the data after it has been retrieved from the server. This can be useful for example to wrap your Model objects into a ViewModel before insertion or to do grouping.
  • ISupportIncrementalLoading for Windows Store apps. Some controls in the Windows Runtime support an interface called ISupportIncrementalLoading. This interface allows controls to request extra data when the user scrolls. We have built-in support for this interface for Windows Store apps. Our windows store extension assembly contains the MobileServiceIncrementalLoadingCollection which automatically handles the calls from the controls.

To use the new collection you can simply use our ToCollection() extension methods on IMobileServiceTableQuery<T> and IMobileServiceTable<T> or use on of the available constructors for more customizability.

To actually load data you have to call LoadMoreItemsAsync(). When you have specified a page size before, the collection will only retrieve so much items as you specified, otherwise it will just send the original query to the server. Make sure you always await the calls to LoadMoreItemsAsync and wrap it with a try { } catch { } to handle any exceptions from the server.

MobileServiceCollection<TodoItem, TodoItem> collection = await client.GetTable().ToCollectionAsync();

or with page size 10:

MobileServiceCollection<TodoItem, TodoItem> collection = await client.GetTable().ToCollectionAsync(10);

Note that to load more pages you have to call LoadMoreItemsAsync() yourself.
If you want to do paging in a Windows Store app, you can also use the MobileServiceIncrementalLoadingCollection<TTable, TCol> which implements ISupportIncrementalLoading. For this class you cannot specify a page size since the controls automatically request a number of items. You also do not need to call LoadMoreItemsAsync() since the control will call this method for you.

If you want to do paging on a platform other than the Windows Store you currently don’t have ISupportIncrementalLoading. Instead you have to wire this up yourself. For Windows Phone you can use Scroll Compression states and call LoadMoreItemsAsync yourself.

HttpClient

In this release we are replacing Service Filters, on the Managed C# client, with HttpMessageHandlers which are part of HttpClient. HttpClient offers several advantages over our previous implementation and HttpMessageHandlers allow developers to reuse their handlers for other projects using HttpClient.

ServiceFilters can be easily transformed into HttpMessageHandlers. Consider the following filter created by Josh Twist:

public class BusyServiceFilter : IServiceFilter
{
	private int _callCount = 0;
	private Action _busyIndicator;

	public BusyServiceFilter(Action busyIndicator)
	{
		_busyIndicator = busyIndicator;
	}

	public IAsyncOperation Handle(IServiceFilterRequest request, IServiceFilterContinuation continuation)
	{
		return HandleAsync(request, continuation).
			  AsAsyncOperation();
	}

	private async Task HandleAsync(IServiceFilterRequest request, IServiceFilterContinuation continuation)
	{
		// update the count by one in a single atomic operation. 
		// If we get a 1 back, we know we just went 'busy'
		var outgoingCount = Interlocked.Increment(ref _callCount);
		if (outgoingCount == 1)
		{
			_busyIndicator(true);
		}

		IServiceFilterResponse response = await continuation.Handle(
			request).AsTask();

		// decrement the count by one in a single atomic operation.
		// If we get a 0 back, we know we just went 'idle'
		var incomingCount = Interlocked.Decrement(ref _callCount);
		if (incomingCount == 0)
		{
			_busyIndicator(false);
		}

		return response;
	}
}

This filter creates a busy indicator based on in-flight requests. It can be easy transformed into an HttpMessageHandler. The following sample show an HttpMessageHandler which does the same. Custom HttpMessagehandlers should in most cases derive from DelegatingHandler as they are meant to be “pass-through”.

public class BusyHandler : DelegatingHandler
{
	private int _callCount = 0;
	private Action _busyIndicator;

	public BusyHandler(Action busyIndicator)
	{
		_busyIndicator = busyIndicator;
	}

	protected override async Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	{
		// update the count by one in a single atomic operation. 
		// If we get a 1 back, we know we just went 'busy'
		var outgoingCount = Interlocked.Increment(ref _callCount);
		if (outgoingCount == 1)
		{
			_busyIndicator(true);
		}

		HttpResponseMessage response = await base.SendAsync(request, cancellationToken);

		// decrement the count by one in a single atomic operation.
		// If we get a 0 back, we know we just went 'idle'
		var incomingCount = Interlocked.Decrement(ref _callCount);
		if (incomingCount == 0)
		{
			_busyIndicator(false);
		}

		return response;
	}
}

Breaking Changes

The switch to HttpClient, the Portable Class Library and the new collection are breaking changes compared to our previous client, but we believe they all benefit developers.

Other breaking changes we made include a switch to Json.NET. While we did our best to keep the behavior the same, there are small changes that could affect you. Most of them actually make your life easier. For example Uri instances can now be serialized without a custom converter.

We also fail early when you try to send numbers, such as longs, ulongs and decimals, to the server which can’t be handled by the server because the number is too large or needs too much precision. As most of you will know, our server runtime is running on JavaScript, in which every number is a double. This means for example you can’t send a long to the server which is greater than 2^53.

exception

Our behavior with multiple calls to Skip(int) and Take(int) has also changed. Multiple Skip calls will now add up, so Skip(10).Skip(8) will result in Skip(18). Multiple calls to Take(int) will result in the lowest being used, so Take(2).Take(30) will result in Take(2). This makes our API consistent with other OData client libraries such WCF Data Services.

More about breaking changes here.

Pre-release

As mentioned before, this is a pre-release of our new library. Things are likely to change between now and the stable release, although we always try to keep the public changes at a minimum. We do NOT recommend the use of this library if you have critical business depending on it, as it is likely to have bugs and is not officially supported.

This release is meant for testing purposes and for us to get feedback on the changes we made. Try it out if you have the chance and submit your feedback and bugs via the forums!

NuGet

This pre-release client is only available on NuGet. Did I already mention you need NuGet 2.1 for it to work? Otherwise Portable Class Libraries and Windows Phone 8 won’t work. If you receive errors such as:

6507_NuGetFailedNeedsUpdate_thumb_3C41099C

Then you need to update your NuGet version via Tools -> Extensions and Updates.

8662_ExtensionsAndUpdates-2_thumb_3BD4D6A7

What’s next?

We plan to update the NuGet package regularly with the latest bug fixes and improvements. Stay tuned for more things to come in the months ahead!

Johan Laanstra (SDE Intern, Windows Azure Mobile Services)

Did you like this? Share it:
14Jul/111

Building the City Cloud part 1: Overall system architecture

The last couple of months I've been working on my bachelor project to finish my bachelor computer science. Together with Tom Verhoeff, Jos Kraaijeveld, Jochem Toolenaar and Oana Nitu, I participated in the Imagine Cup 2011. We formed team O!ife and our project was called the City Cloud. The City Cloud is a cloud computing platform that allows different kinds of data generated by the city, its inhabitants, companies and its government to be easily accessed by developers to create new innovative technologies and solutions. You can find more about the City Cloud on the O!ife blog.

This post is one in a series on how we built the City Cloud and everything around it. The focus of these posts will be on the technical aspects of the City Cloud. It will also cover some problems we encountered during development and the solutions we used to tackle them. At this moment I have no idea how many parts will follow. We've came across lots of different problems and used lots of technologies, so I will write another part whenever I have the time and when I think the topic is interesting enough for other developers. The first part will cover the overall architecture of the system to establish a background for other posts.

Goal

When we started the City Cloud project we thought of it as a developer platform for a city. The main goal should be to offer the correct tools for developers to build next generation ubiquitous applications: it should be easy to expose new datasets through the system and the development of applications using these datasets should not require large investments in both terms of infrastructure and programming skills.

In the design phase we divided the system up into three parts: the cloud framework, the mobile platform and the website/webapplication. The framework was the heart of the system and was designed to run in the Azure cloud. Its responsibility was to present the data that is part of the system in an easy and consistent way and offering an all-in-one solution for data and app storage. The mobile platform and the website/webapplication functioned as two different portals to the data and the apps. The website should give general information on cities and should show the datasets and applications available. It should also provide developers with documentation and best practices on how to make use of the platform's capabilities in the best possible way. The mobile framework should be the main portal to the end user in a city. It should give an user specific information on data and apps based on his current location. With more and more smartphones sold every day, smartphones offered a great opportunity to be the number one entry point to the City Cloud.

Framework

The framework should expose data in an easy way. To achieve this we decided to go with an OData webservice running in an Azure webrole. Besides being a Microsoft technology which counts for the Imagine Cup, OData offers an excellent way to expose datasets and besides that offers great possibilities to perform transformations and filtering on the returned data. Another advantage is the great tools available to develop OData webservices and to consume them on almost all major platforms. To get data into the City Cloud framework we invented "connectors". To expose a new dataset through the framework all a developer would need to do is build a connector that allows the City Cloud to retrieve data from their data source. This connector should implement some interface, provide the system with some description of the data structures and it should all magically come to life.

Mobile platform

To reach as much as possible users we wanted the mobile platform to run not only on Windows Phone, but also on Android and iOS. But we didn't want developers to write the same application three times for three different platforms. To accomplish this we decided to go with html5 and javascript as the tools to build the applications for our mobile platform. On every major smartphone platform it is possible to embed some kind of web browser control into a native application and load webpages in it and with the Mango update for Windows Phone coming this fall, html5 is also broadly supported on all mobile phones. We call these html5 applications "building blocks". These building blocks could be hosted in the City Cloud framework and could use datajs to interact with the OData webservice via javascript.

Website and webapplication

To give developers and users an overview of the data and applications already in the system we built the website including a Silverlight webapplication. The webapplication was meant to be used as an explorer for the webservice. In this way everybody could take a quick look into the system to browse data. It also offered the possibility to plot these datasets on a map.

Extensibility

Developers could use the website to submit new applications and new connectors to the system. Because of the extensibility model we felt the need to building a validation step to verify the working of the newly submitted parts in the framework. To validate new submissions we made use of a core feature of the azure platform: a worker role. In Windows Azure a worker role is meant to be used for background processing or other long running tasks. In the cloud framework we used a worker role to validate a submission and to make the required changes to the framework to add a new connector or building block.
At the end of the design phase the overall system architecture looked a lot like a high level Model-View-Controller architecture. The website and the mobile platform functioned as the views, while the framework holds the current state of the system and functioned as the model. The worker role was used to change the state of the framework and can thus be seen as the controller. The following image shows the high level architecture in a graphical way:

I hope this gives a good overview of the system we built the last couple of months. Unfortunately no code yet, but I will save that for the next part.

Did you like this? Share it: