Recently I had the privilege to make my sixth appearance in the CRM Rocks podcast run by Markus Erlandsson. Between our first session back in October 2013 and today, the world of CRM has certainly gone through a lot of interesting events. It’s amazing that out of those ~2600 days in between, there are now more days when a product named “Dynamics CRM” has NOT existed than it has.
CRM of course still is very much a “thing” – it just isn’t the center of the Business Applications universe anymore. In our podcast with Markus we talked about the broader scope of what has happened with both the first-party Dynamics 365 applications as well as the Power Platform in the year 2020. Put on your headphones and listen to the CRM Rocks episode right here.
The topics in the podcast also tie in nicely with another tradition of mine, which is writing these end of the year retrospectives about the top 3 topics that I’ve found to be most interesting or impactful for our ecosystem. Before we get started with the fourth edition in this series, let’s quickly check back on what themes I picked out last year at the end of 2019. They were: 1) low-code movement, 2) licensing confusion, 3) data story evolved.
All fairly hot topics still today – which is of course the reason why I emphasized their importance one year ago. Now for 2020, this time I’ll keep you on the edge of your seat by introducing the top 3 in reverse order. Hold on tight, here we go!
Nr. 3 from 2020: Renaming frenzy
We all know that Microsoft loves updating not just their software but also the terminology used for referring to these products. CRM is of course a prime example of this. Microsoft’s isn’t always “following the money” by sticking to industry standard names, rather they prefer to reimagine the categories in which their technology can be used for solving new business problems.
Be sure to pass along that advice to branding folks at MS who decided to deprecate names like "CRM" and "Office"😆
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) December 9, 2020
In 2020 we saw the iconic Office 365 brand get removed from the official vocabulary of Microsoft’s list of products available for purchase. Like with CRM, the actual technology and the customer use cases for the products didn’t see any major overhaul overnight. Previously there were SKUs in the price list branded Office 365 and Microsoft 365, which caused confusion for the customers, according to Microsoft’s own branding change announcement. Luckily now everything is crystal clear and no one will be talking about Office 365 in the year 2021 anymore. (Right?)
While Offi… sorry, I mean Microsoft 365, is a mainstream product used by pretty much all customers in the MS Cloud, its rebranding in Spring 2020 turned out to be just a warm up for the real main event of 2020 that took place in July. I’m of course talking about the rise and fall of Microsoft Dataflex. In may have only lasted for exactly 3 weeks but the Dataflex era will surely be remembered as the crazy days of software branding.
Here's why Microsoft has felt the pressure to remove the word "Dataflex" from its websites, as (rightfully) claimed as a trademark infringement by Data Access Corporation: https://t.co/s2RXCPTl5Ypic.twitter.com/F0fufjBTPo
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) August 11, 2020
Luckily the Dataflex saga started in the middle of the year, which means we were able to catch the season finale before 2020 ended. The revised names of “Microsoft Dataverse” and “Microsoft Dataverse for Teams” were revealed in the GA launch of what used to be codename Project Oakdale. These names have persisted for six weeks already without changes, so they seem to be the real deal now.
Following along the path travelled by CRM and XRM earlier, the three letter acronym of CDS is therefore now officially deprecated. What were perhaps more shocking casualties of the year 2020 were the terms “entity”, “field”, “record”, “option set” and “two options”. (I’m not counting “multi select option sets” in this list since everyone who’s had to deal with them probably hated them already, so good riddance.)
As for how many adjustments there were to products in the Dynamics 365 family, I honestly can’t even tell. The fact that I needed to do a dedicated “why MS product naming is complex” type of a blog post to analyze the phenomena should be firm enough evidence that the renaming frenzy definitely was one of the top themes of 2020.
Nr. 2 from 2020: Power Platform licensing complexity growth
Measured by my own articles and the comments I’ve been hearing, there seems to be no end to how much headache the licensing of different Power Platform components is causing to people. Even though most of the last major changes to the licensing model were actually introduced already in 2019, I think the impact from them has only now started to hit home with a lot of the technical folks when trying to apply the new rules in real life scenarios.
Earlier in the year there was a lot of fuss around the coming technical enforcement of access rights on the app module level. The immediate attention was of course on the arrival of Dynamics 365 Team Member licensing enforcement and the targeted app modules offered by MS while they would be blocking all other standard and custom apps. This was supposed to take place already on April 1st, but then COVID-19 came around and the deadline was pushed to January 31, 2021.
We did still see a growing number of technical limitations implemented to keep customers in compliance with the licensing terms while using MS cloud services. The CDS Dataverse storage capacity in particular became a big practical blocker for customers wanting to leverage multiple Power Platform environments to some ALM practices for their low-code apps.
While storage consumption is already enforced today, the big question on every developer’s mind (well, at least those who follow what the Power Platform community talks about) must be the API call limits. We’ve been getting a few bits & pieces of information here & there from MS about how they actually plan to keep track of the API quota for different operation types, but no definitive guide exists today. The eventual enforcement of these rules could potentially lead into future troubleshooting patterns where we could replace “DNS” with “licensing” in this popular meme:
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) October 21, 2020
In the year 2020 I wrote in total 9 blog posts on the topic of Microsoft Business Applications licensing – including a 4 part series on the various sources of complexity in licensing. As for Microsoft, during the year 2020 they didn’t manage to launch the licensing consumption metrics dashboard that has been promised for Power Platform Admin Center ever since the October 2019 licensing model came into effect (on paper). With all these factors combined, licensing complexity in my eyes earns again a place in the 2020 Top 3 themes, just like it did in 2019. Let’s see if in 2021 things will settle down a bit and we’ll finally gain more confidence in understanding and managing the licensing costs of solutions built on top of Power Platform.
Nr. 1 from 2020: Microsoft Teams as a platform
There’s not a single doubt about what the year 2020 will be remembered for on a global scale. The total impact from the COVID-19 pandemic reaches far beyond the mere side effect that it had on us information workers, with everyone being forced to join the #WFH trend and getting used to all of our work being remote work. Still, that is the most immediately and easily observable part of what the virus did to us. As a result, for pretty much anyone working in the MS ecosystem, 2020 became the year of Teams, Teams and a whole lot more Teams.
Working from home proved to information centric businesses that a surprisingly high percentage of those processes that weren’t fully digital before COVID-19 struck could actually be turned into such when no other option was given. While this will give a major boost to future digital transformation investments for sure, it also demonstrated how little new tech was actually need at the end of the day to make the typical business processes work without any sort of physical encounter. Already with our collaboration tools of 2020 and the low-code application platforms of 2020, many teams have learned to creatively solve their problems and get the job done. No massive multi-year initiatives and agile software development projects with big budgets were needed to reach “good enough for now”.
In 2019 Microsoft started promoting Power Platform as “the platform for every developer” by addressing the pro developer audience on the benefits they could reap from adopting low-code tools as part of their app development toolkit. In 2020 it was time to address a much wider audience of power users, by promoting Microsoft Teams as a platform for the apps they could be creating for themselves, for their teams, or for their entire organization.
Power Platform as a whole has managed to reach 1o million monthly users, based on the latest Microsoft FY21 Q1 earnings call. While that’s an impressive figure, it’s nothing compared to the 115 million daily active users that Microsoft Teams has. In my own analysis on this Teams as a Platform motion, I wrote about how Teams is gradually transforming into the next Windows:
Teams is now the closest thing that Microsoft has at its disposal to transform into an OS style fabric that connects a significant share of information workers globally.
By bundling the basic Dataverse functionality inside Microsoft Teams both in technical and licensing terms, I’d say we’re seeing the low-code equivalent of what the Internet Explorer + Windows bundle represented back in the early days of the commercial usage of the web. It will be pretty tough for the competing vendors to find their way inside any existing corporate IT infrastructure on the same breadth as what Teams has already achieved.
Teams of course was delivered there as part of the licensing payload of Office 365, which lead to competing workplace messaging platform Slack to file an antitrust complaint against Microsoft in the EU in July 2020. Later in December when the business app platform vendor Salesforce acquired Slack and announced it plans to use it to “create the operating system for the new way to work”, this kind of validated the strategy that MS had followed with Teams all along.
The way I see it, Business Applications will increasingly be presented within the context of where the work actually gets done. They will become more targeted for the specific user groups and use cases that may well be unique to every organization. The complex ERP style systems of record will still exist behind the scenes, to orchestrate how work gets assigned, scheduled and invoiced. For the actual end user experience of interacting with these work processes, though, it will likely be closer to just using an app inside Teams than what is required operating the complex UIs of traditional enterprise applications. This is why the biggest theme of 2020 in Microsoft Business Applications has to be the concept of Teams as a key layer in the application platform story.
After the peculiar episode in July-August timeframe when Microsoft first tried to rebrand the Common Data Service and had to later cancel their plans, we’ve been eagerly awaiting to hear what they are actually going to call CDS from now on. Yesterday the Teams based features known as Project Oakdale finally went GA and now we know the answer. I won’t focus here on the actual features, but you can read my thoughts on them from the Forward Forever team’s blog:
The mapping of the old & new names is quite straightforward:
CDS = Dataverse
Project Oakdale a.k.a. “CDS Lite” = Dataverse for Teams
Not all that different from their initial plans of using the name “Dataflex”, which didn’t go through since the owner of that trademark considered it to be an infringement and sued MS. Is the name “Dataverse” unique then? Not entirely, as there appears to be at least the Harvard Dataverse out there. Luckily it’s not as directly related to low-code application development platforms as the DataFlex that caused grief the last time.
Just because there are other existing products out there in the IT space that use a particular word, this doesn’t mean new products would never appear in the market with the same name. As an example, last month a service called Microsoft Clarity was made Generally Available. Is “Clarity” a brand name that isn’t used by any other IT product? Of course it isn’t. The first thing that comes to my mind is Clarity PPM project & portfolio management software, but search engines will reveal many others. Heck even MS has acquired a BI software company called ProClarity back in 2006.
That’s just how it goes in the real world. Coming up with a name that hasn’t ever been used before yet also means something is not an easy task. If instead of a Globally Unique ID you’re aiming for a product name that actually contains some hints about what the product does, then your choices will always be limited.
Not only does the name of the technolgy itself change from CDS to Dataverse, there have been a number of terminology updates to the foundational concepts of the data platform:
Entity > Table
Field / Attribute > Column
Record > Row
Option set / Picklist > Choice
Multi select option set > Choices
Two options > Yes/No
For a quick explanation about the scope of changes, see this short video by David Yack:
I bet this is a bigger change for many of the professionals in Power Platform and Dynamics 365 area than the new Dataverse brand. We’ve been talking about entities ever since the birth of MS CRM and now all of a sudden the idea of “entities are much, MUCH more than a mere database table” should be replaced with “fine, they are tables”.
Why are these changes necessary then? Largely for the same reasons why a more approachable name was needed for CDS: the target group for this technology is no longer the same – it is considerably broader. Microsoft doesn’t want the platform to be an exclusive club for those who thoroughly understand the business process management and data modeling techniques offered by their products, sometimes only after having gone through training courses and technical certifications. No, this needs to be something a power user that’s equipped with nothing more than a Microsoft Teams license can quickly figure out on their own.
The path towards Teams as a platform brings more changes than just new names for familiar things. Previously the data model construction was a back-end task handled largely separately from configuring the app’s front-end. With Dataverse for Teams and later presumably in other UI’s, too, the interactive table editor supports both data model design and data entry all in one – much like a pre-built app á la Microsoft Lists does today. It doesn’t remove all the other functionality of entities in the underlying relationship management system, but it makes these optional at the start. You could just build a table like in Excel or Access, then later take it further as the app requirements become better defined.
Earlier in XRM everything was more abstract, whereas now we're going towards an actual visual table with columns when building a CDS data model. I gotta admit it would be pretty bizarre to tell the new users "and in this UI right here we'll define attributes for an entity": pic.twitter.com/fy5EEECY0H
For those who have mastered the earlier XRM universe, the simplification of the terminology that was very specific and accurate to what the platform does may feel like a slap in the face. How can Microsoft tell us that we should pretend that it’s just a database table when we know the reality as well as the history? There isn’t much of an upside for these technical professionals in the new naming – at least if they don’t need to interact much with business users unfamiliar with the core concepts of CDS.
We’ll get over it, though. Developers and even many no-code customizers will probably hold on to the entity/field terminology for quite a while – especially when on the API level everything the terms are not expected to change (at least for now). Having this dual reality of tables and entities isn’t going to present a huge cognitive challenge for those who’ve been with the platform for a longer time. The old content from pre-Dataverse era may cause confusion for the newbies, but we’ve already been through the merger of Power Apps & XRM a short while ago, which had a much more profound impact on the names of concepts in the platform. This too shall pass.
Community Confessions: Do you really care of #Microsoft changes terminology in the #PowerPlatform?
While it may not be of much consolation, there’s one thing that we know won’t change: this changing of product and feature names isn’t ever going to stop.
Why is this naming thing so hard in BizApps?
I’m not going to write a four-part series on the product naming complexity just yet (like I did on the complexity of Power Platform licensing). Let’s try and keep it within this blog post for now. So, here are four reasons that make naming especially tricky in the context of Microsoft Business Applications are:
Ambiguity
Hierarchy of concepts
Shifting product boundaries
Marketing goals vs. market needs
1. Ambiguity
In a universe as broad as Power Platform, there can easily be overlap in naming elements of that platform that serve different purposes but really should have a fairly generic name. As an example, “a Power Apps portal” and “the Power Apps portal” are 2 different things. Excuse me, what?!? Yes, it’s all in the documentation. Power Apps portals are one of types of apps you can build, whereas the Power Apps portal is the place to create new tables and columns. People tend to refer to the latter one as the Maker Portal thanks to its URL, but it’s good to keep in mind that URLs in Power Platform do also change occasionally.
This is why we now have Model-driven app forms which use the word “column” for both describing a field on a form as well as the number of adjacent containers within a tab or section. The original intentions for using easy and generic terms can sometimes create confusing results. If not in their native English language then most probably somewhere down the line when the terms get localized to a multitude of different languages.
2. Hierarchy of concepts
You won’t always have a clear view of what individual products and technologies will eventually be needed when pursuing a particular vision. Thinking about what CDS was called initially, it wasn’t a separate “thing”. It was embedded within the Common Data Model concept introduced in 2016. It took a year for this vision to evolve to a level where the actual data platform functionality operated by Microsoft was named Common Data Service and the earlier CDM term was dedicated to the idea of a shared schema across business apps from different vendors. It’s easy to see these should be 2 different things, but finding the right way to put them out there took a few iterations.
Another example where things are a bit further already when a the need for redefining the terminology arises is Customer Insights. Microsoft has been shouting a lot about the booming demand for their Customer Data Platform solution (CDP, a generic term like CRM). Recently they’ve had to reshuffle the Insights deck of cards to ensure the gameplay works smoothly, so a hierarchy like this has been introduced:
Customer Insights: the top level concept of Microsoft’s CDP
Audience Insights: the features that were initially called Customer Insights, now referred to merely as a “capability” of CI.
Engagement Insights: announced originally as an independent product called Product Insights, now in preview as another capability of CI
3. Shifting product boundaries
Microsoft today excels in understanding the importance of their technology stack. You could also call it the platform approach in this context, but the point remains the same. Their products rarely get developed in isolation these days, rather there is (at least seemingly) a big push for combining elements of existing services to create new products. The problem is, though, that this game of connecting the dots isn’t ever really over.
Survey feedback management was first brought into the Business Applications portfolio via acquiring an ISV product called Mojo Surveys that had been built on the XRM platform. After some rearchitecting and rebranding it was launched as Microsoft Voice of the Customer (VoC). The capabilities for the survey form design were quite lacking compared to the simplified UX in Microsoft Forms that had been developed internally on the Office 365 side of the house. Instead of retrofitting the form designer into VoC, MS decided to take CDS as the process and data management engine to give birth to a more structured feedback management tool, called Forms Pro. VoC was then deprecated and customers were asked to migrate to Forms Pro instead.
In this process the technical capability of feedback management had hopped over the fence from the Dynamics side to the Office side. This was somewhat problematic from a business perspective, as well as causing confusion between the non-Pro and Pro usage of Forms. So, Microsoft ended up doing a further reimagining of EFM and rebranded Forms Pro to Dynamics 365 Customer Voice, bringing it back to the BizApps side where it started from.
Returning to the previous example, before we had Dynamics 365 Customer Insights, the product names Microsoft used for referring to the same concept included “Cortana Intelligence Customer Insights” and “Azure Customer Insights”. The earlier Azure products that were offered in preview had to be scrapped and rebuilt as a Dynamics 365 product to finally see the GA milestone. Today it makes far more sense to have these type of packaged AI capabilities in the BizApps portfolio than in the Azure service catalog, but it wasn’t always obvious – especially back when Dynamics was just about CRM and ERP.
4. Marketing goals vs. market needs
Product marketing at Microsoft is in an endless pursuit of finding the best story that can sell customers a shiny new vision of what results they could achieve – if they choose the right technology from MS. Making a single purchase isn’t enough. You have to get the customers hooked on buying more from you, which leads to the invention of new things to sell. It’s easy to understand the incentives that lead to such actions, but they can sometimes end up irritating the customer who’s already made a purchase based on their own understanding of their needs.
When Microsoft decided in 2016 that it would no longer sell a product called CRM, you could argue that their vision for tearing down the traditional CRM and ERP silos was logically a goal worth pursuing. The side effect from this product marketing goal was that when Microsoft took away the name “CRM”, they didn’t give anything new to replace it with. Reluctantly the Customer Engagement name was introduced a bit later, only to be quietly discontinued.
The CRM systems that customers had deployed didn’t disappear anywhere of course. Companies offering professional services around them first struggled in migrating from Dynamics CRM consulting to Dynamics 365 CE consulting, and now they’re even more confused about what to call themselves.
Microsoft’s branding changes aren’t ever exactly applauded in the partner or customer space, since renaming CDS to Dataflex to Dataverse will always cause extra work for them and give little in return. Removing an entire category like CRM has a far bigger impact than changing A to B. Redefining concepts and their boundaries is of course a very powerful way to stir things up and generate motion in the markets. I can’t say that I personally have any regrets on moving on from CRM, yet it’s easy to understand why not everyone is cheering when someone comes and says “what you’re doing is no longer a thing – time to reimagine everything!”
At the virtual Microsoft Ignite 2020 event I had chance to join many fellow Finns in our FIgnite session and spend 5 minutes on highlighting a topic that I considered to be important in the MS ecosystem right now. It turned out to be an easy choice to make.
Project Oakdale (briefly known as Dataflex) is bringing the low-code/no-code functionality from Power Platform into the hands of pretty much all the information workers leveraging Microsoft 365 tools in their day-to-day job. This is not so much about any single technical feature as it is a change in mindset about who the app makers should be. So, I set out to explain this my 5 minute presentation titled How Power Platform Now Empowers All Teams Users.
https://www.youtube.com/watch?v=Dj_IH4yq1dE
This is all part of the larger Microsoft Teams as a platform story that I covered in my previous pre-Ignite blog post. It’s no surprise that MS has created much more content around this topic than you could have ever expected to see on Power Apps or Power Automate as independent technologies. It’s a sign of the size of the audience to which this “no cliffs” application platform message is now targeted at.
Will the inclusion of “CDS Lite” in the Microsoft Teams subscriptions prove to be a gateway drug that makes Common Data Service a household name in Microsoft 365 customer organizations? And if so, what will the actual brand name be for Project Oakdale once wre’re past the preview phase? Time will tell!
2020 became the year of #WFH (work from home) and for many organizations also the turning point when Microsoft Teams became the primary place where being “at work” happens. This is accelerating the evolution of Teams from being merely a communication tool that connects human beings into a foundational service layer for many types of business applications.
How the concept of Teams as a platform contrasts with Microsoft’s Power Platform suite of technology is something I’ve been thinking about a lot lately. In this post I’ll first reflect on the relatively short history of where Teams came from. I’ll then examine how the recent feature announcements are brining apps front & center in Teams. Finally, a few words on the possible future for Teams as part of Microsoft’s broader strategy.
The road that lead to Teams
Looking back ~10 years, the real-time communication & instant messaging tools from MS seemed to be going through an endless renaming cycle: from OCS to Lync to Skype for Business. The core feature set presented to the end user didn’t seem to evolve nearly as much as product branding did. On a broader level, the communication activities of information workers within an organization still typically took place within Outlook’s inbox, and different servers like SharePoint and Dynamics CRM all packed their own features for posting short messages to other users.
4 years ago, when the first images of what was then called “Skype Teams” started to leak out, we were already waiting for MS to create something a bit more ambitious than just another online meeting tool. Office Groups had began to emerge in various different places inside the MS Cloud, but they were primarily a technical construct with no sensible UX for everyday people to approach them. Even Dynamics CRM had it’s own solution that attempted to bring together the dicussion, calendars, notes, documents and team memberships from under an Office 365 Group associated with a record like account or opportunity:
I remember having many discussions with our CRM customers where I attempted to steer people away from deploying this Groups solution. Instead I wanted to encourage them to wait for something a bit more polished that I knew had to be on it’s way sooner or later.
At one point there was a clear & present danger of another “Yammer moment” taking place, as Microsoft was reportedly quite serious about their plans to acquire Slack. In retrospect it was a blessing for both parties that MS decided to keep investing in building their own product, instead of trying to retrofit an established service like Slack into their existing software offering.
I would argue that this “build over buy” strategy which Microsoft has since then followed across their business software stack has been a key success factor for BizApps in particular. It has enabled MS to move from merely chasing CRM competitors like Salesforce into redefining the business apps playing field with Power Platform. There’s a stark difference between acquiring companies and bundling them as “X Cloud” versus engineering your own software stack to act as a true platform.
Teams: the collaboration chapter
Initially the first version of the Microsoft Teams product that became generally available in Spring 2017 was pretty much focused on being three things:
Replacement for Skype for Business
Alternative to Slack
UI layer for Office 365 Groups
From a business applications perspective there wasn’t all that much you could do to hook Teams up with Dynamics 365, until Fall 2018 when the previews for the first integrated features were launched. In particular the integrated file sharing experience that Teams offered seemed almost like the Holy Grail for many CRM professionals, offering to fix the glaring hole in the SharePoint integration story that lacked any security model synchronization. The roadmap image below presents the plans from 2 years ago on how Teams and Dynamics 365 were going to be integrated:
The last item on the roadmap has still not been delivered, which is the visibility of Teams conversations inside the Dynamics 365 record form. Why this hasn’t been a higher priority for MS to implement seems to me like a sign of how Microsoft Teams is nowadays positioned as the primary UI for all information work. MS probably would prefer if everything always started from inside Teams. You pin record tabs into channels, you show previews of records inside teams discussions, you interact with records via bot interfaces and so on. As long as Teams is that big umbrella under which all work takes place.
The lack of a deep 2-way integration does not therefore mean that investments aren’t being made into the products involved. It can simply be a reflection of the new vision that is being built, by aligning many existing services to form a whole that aims to be greater than the sum of its parts.
As an example, if you look at Microsoft’s task management story, you’ll see that features and data from across various apps like To Do, Planner and Outlook tasks / flagged emails are currently being collapsed into a central location that is the Tasks app for Teams. Tasks as a generic construct don’t necessarily need to be fully controlled by a single database, yet they very much need to be logically represented within “the hub for teamwork” that Teams is positioned as.
Going forward, when new apps appear into the MS cloud product portfolio and they need to offer task management features to users, the logical integration point to focus on would be Teams. For activity feed type of functionality the choice is even more clear for product development: choose to piggyback on Teams instead of inventing yet another stream of short messages.
Teams: the platform chapter
Moving beyond simply integrating Teams with products X, Y and Z, we’re now seeing the rise of a model where apps are built specifically to be used in Teams. This has of course been possible for a long time already, by developing custom web services and using the SDKs. Now there are many features coming up that will amplify the platform story around Teams on the no-code/low-code front specifically.
Microsoft Lists app has been the first to reach GA and offers an ultra low barrier for users to process data in a single table through a configurable, readymade UI. When accessed via Teams, the list data gains one more special dimension: discussions to be had regarding a list item. This is pretty much the same as the usage pattern offered for a Dynamics 365 record with the integration mentioned earlier.
Underneath the new covers of MS Lists is the technology familiar from SharePoint lists. If we were to only examine the UI layer, there is actually a remarkable similarity to a popular no-code service called Airtable. So much that the accusations of MS simply copying the visuals and core features from this competitor don’t seem entirely unjustified.
Comparing these two offerings gives us some perspective on what exactly is the market position these tools are aiming to conquer. Simple lists themselves are not a particularly unique feature, rather it’s the team collaboration capabilities and ease of data sharing that turns these tables into what you’d call an actual app. Incidentally, just this week Airtable announced they were building a full platform with apps offering JavaScript based extensibility, a marketplace for sharing apps, automations for executing business logic, and finally a sync service to transfer data across environments (“bases”).
Collaboration scenarios around semi-structured data like lists and Excel style tables can be seen as a gateway drug. They allow turning email or paper based manual processes into a quick first draft of what the digital process could be like. If there are indeed clear business benefits in automating the said process, the requirements for more complex app features will soon begin to emerge from the user base. Hence the collaboration platform should offer an obvious path to grow these pre-built app experiences into more advanced no-code/low-code apps.
Project Oakdale a.k.a bringing CDS to Teams
If Microsoft Lists is the equivalent of an Excel table within the Teams context, then Project Oakdale / “CDS Lite” could be though of as bringing SQL Server inside Teams. Now, obviously Microsoft has zero intent on actually replacing Excel nor SQL with features built into Teams. They only need to introduce those parts that make sense from a team collabocation perspective.
Microsoft Lists is a far cry from what a real Excel workbook can do, yet it can offer much more value in a collaboration scenarios that those lone .xlsx files ever could. Similarly, the version of CDS that will very soon be available for building Power Apps within Teams is nowhere near as powerful as the services powering enterprise CRM systems like Dynamics 365 (or the raw power offered by SQL). Still, the fact that it can be found from within every team and used by a much larger audience than what Power Apps citizen developer tools could hope to capture – those are the factors that can truly make CDS a mainstream service that most information workers in the Microsoft 365 cloud interact it.
The experience of defining the CDS data model in Project Oakdale will be very different from the path that Power Apps makers have gone through – let alone the XRM veterans. In fact, you could easily mistake the table design and row entry UX to be that of Microsoft Lists rather than CDS. This highlights a key aspect that not all Power Platform experts may yet have grasped: for MS this “CDS Lite” is not so much about deciding what premium features of the full Power Platform to give away for free to Teams subscibers – rather it’s about how to best simplify the enterprise CRM features of CDS into a new product that Teams users could adopt on their own.
This doesn’t mean that Microsoft Teams should be viewed only as a mechanism for MS to scale Power Platform to the masses, by “dumbing it down”. If the app platform story of Teams plays out like it ought to, there should also be clear benefits from it to enterprise business applications development.
Capabilities like messaging, notifications, task management, documents or group memberships are not something the Power Platform tools are very good at. For historical reasons there has been the need to build standalone features into XRM for these type of common requirements found in business application scenarios. For the future generations of apps being created, it’s easy to see the benefits of having these non-core capabilities offloaded onto a platform more suitable for managing them – meaning Teams.
It doesn’t really even matter if the feature set offered by Teams couldn’t cover all the deep business logic integration of native Dynamics 365 functionality. Ultimately it’s not about supporting the system-of-record legacy but rather encouraging the new low-code scenarios that will generate 100x more apps onto the platform.
Teams is the new Windows?
The concept of an operating system is something many of us may relate back to the origins of the Personal Computer era, even if OS’s of course have existed far longer than the IBM PC. Windows was the first runaway success in the OS space when it comes to both awareness and commercial results, shaping the fate of Microsoft for roughly three decades. Then along came the era of mobile computing and Android & iOS took over in the number of devices running them. MS could no longer hope to regain that position so they decided to take over a different layer in the computing space: the (business) app cloud.
Azure has been called “the world’s computer” and this does offer some perspective on how the computing concept has evolved since the PC days. Still, Azure is not something most people will ever interact with directly. To remain relevant in the decades to come, MS needs to have presence in the minds of the end users, too. Now that Windows has become merely an optional part of the modern computing stack, it would be pretty darn critical to gain a strong enough foothold on a level that’s above the traditional OS but still below the individual apps. A platform that spans across all the devices people in the business world are using.
Teams is now the closest thing that Microsoft has at its disposal to transform into an OS style fabric that connects a significant share of information workers globally. Nothing like the glory days of Windows, of course, but we should expect to see very conscious steps from MS to further the goal of Teams becoming more OS like. The place where the user interacts with a multitude of apps, share their work context with those apps, a “service bus” for the various apps to exchange data with one another, and finally a unified communications channel for notifications and messaging.
It’s still a long, long way to go for this type of shift to happen where the collaboration tools become the true center of gravity for the multitude of other apps and services that people use today in their #WFH offices. Personally, I can’t live with the limitations on multitasking that the MS Teams content embeds enforce upon the user and much prefer a collection of separate browser tabs to freely switch between. Nevertheless, it’s not an entirely crazy though that the resulting congnitive load for the user isn’t everyone’s ideal way of working. Which means organizations need to be looking for ways to optimize the employee experience via common information work hubs like Teams.
Dion Hinchcliffe has written an excellent analysis on Microsoft’s platform strategy with Teams, where he talks about seeing Teams as the operating system for work. While it may indeed be difficult to get the current owners of the collaboration tools in customer organizations to accept the business app side of Teams onto their plate (especially now with the #WFH boom and its unexpected requirements), the perspectives may change when the time is right from both the technical capabilities side as well as the organizational targets. In the same way as the MS CRM foundation evolved into a key element of the broader low-code application platform known today as Power Platform, the barriers between collaboration tools and business apps should not be perceived to be carved in stone.
There are a lot of benefits in taking long enough holidays during the summer, like many people here in the Nordics do. Going offline for a while is a great way to reset your brain and force-close all those open browser tabs of the mind, to free up memory capacity for properly engaging in brand new tasks.
If you were lucky enough to set your vacation time between July 21st and August 11th, you managed to dodge an unfortunate episode in software product launches from Redmond. Here are the key dates:
July 21st (first day of MS Inspire conference): a new brand name “Dataflex” is announced, with Common Data Service renamed as “Dataflex Pro” and a new subscription tier “Dataflex” promised for all Microsoft Teams customers. (My blog post about it is still here.)
July 29th: the name “Dataflex for Teams” is used for the first time to reference the previous “Dataflex” tier that’s bunlded with Teams, in an effort to make this product connection more clear.
August 11th: all references to “Dataflex” in all Microsoft websites are taken down. The announcement blog posts are brought back online with the terms “Dataflex” and “Dataflex for Teams” replaced with “Project Oakdale”, and “Dataflex Pro” now reverted back to “Common Data Service”.
What happened? The answer is: litigation from the owner of the trademark “Dataflex”.
Here's why Microsoft has felt the pressure to remove the word "Dataflex" from its websites, as (rightfully) claimed as a trademark infringement by Data Access Corporation: https://t.co/s2RXCPTl5Ypic.twitter.com/F0fufjBTPo
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) August 11, 2020
There already was an established software product with the name “Dataflex” out there – in fact it had been around for 4 decades already. Immediately upon the launch day of July 21st several community members were quick to point this out, as indeed it was in practice impossible to miss this brand name overlap when using a search engine. Why Microsoft decided to proceed with their plans on renaming CDS into something like this is something we’ll never know for sure. Some had initially speculated that perhaps there was a preliminary agreement in place with Data Access Worldwide to transfer the ownership of the trademark, but based on the immediate legal action that the company understandably took to defend their rights, nothing like this was likely ever discussed. Our conclusion must therefore be that Microsoft knowingly and intentionally attempted to change the name of its existing technology (Common Data Service) into something that was already in use within the same domain by another software company (Dataflex) without considering the consequences.
The end results is almost like a story taken from The Onion. Even if we are living in 2020 right now and the world appears to go mad every few days, it’s still an utterly bizarre episode in software business. I can’t recall seeing anything killed this fast after it’s announcement, which in the case of Microsoft Dataflex was only 21 days. Neither can I think of an example where something with such a clear potential for software product naming conflict would have been attempted. It is indeed similar to as if MS would have launched “PlayStation 6”.
It’s of course not unusual for Microsoft to pull a product name between its announcement and the planned GA. Remember Dynamics 365 Business Edition, for example? Just because we see something presented on slide decks aimed at MS Partners as guidance for building their business around it, that doesn’t guarantee it will ever see the light of day. Shift happens, plans change and a lot of the partners have already become accustomed to this risk as simply the cost of doing business in this ecosystem. This time the product itself is not pulled, though. The need for coming up with a new brand will most likely delay the commercial roll-out of CDS features for Microsoft Teams based apps, but it’s important to keep in mind that this is all still happening. CDS will most likely become a mainstream data platform to replace use cases where SharePoint Lists have previously been used.
SkyDrive is another example of a Microsoft product brand that suffered from the conflict with an existing trademark and had to be renamed OneDrive as a result. BSkyB as a media broadcaster and digital services provider wasn’t entirely in the same business as Microsoft with it’s personal cloud storage that was following in the footsteps of Dropbox, yet the end result of that case showed that at least between large enterprises such near misses with strong brands can become costly. Even features of a product like the “Metro UI” with Windows 8’s modern user experience had to be avoided due to objections from German reltailer Metro AG. Both of these are examples where it wasn’t obvious to an outsider if a trademark was actually violated, though. With Dataflex the evidence was in plain sight for everyone.
Lucky for us, MS had not yet made any public preview or a commercial subscription of the new Dataflex branded services available by the time they had to call the whole thing off. It was only the early adopters in the partner field and the passionate community members a.k.a. #PowerAddicts that lost some of their work hours due to the branding incident. Yeah, I had to go back and edit a number of web pages, rewrite presentation decks, revert our service documents etc. to recover from the erraneous information that had been published on July 21st. The majority of people out there in the MS ecosystem probably didn’t yet even get a chance to understand what Dataflex meant or how it was different from CDS, so they will have dodged the bullet when it comes to loss in productivity.
There is, however, a much greater loss here than the “Find and Replace” activities needed for changing product names. It is the loss of credibility that Microsoft’s Business Applications specifically suffered from this very unfortunate product naming incident. The worst thing is that there has not been any explanation presented to the genereal audience on what’s going on, rather we’ve just seen the word “Dataflex” disappear and get replaced with “Project Oakdale” or CDS. Microsoft at this point is facing legal action from the owner of the Dataflex trademark, which we all know from the many Hollywood movies we’ve watched that “anything they say can and will be used againts them in a court of law”. It is at moments like these where it would be crucial for companies to have a voice, to be able to reach out to their most important audience (customers) and explain what happened & what are they going to do about it.
I can tell you that it would honestly be a full time job keeping up with A) product naming and B) product licensing of Microsoft Business Applications. It’s not full time for me (luckily), but since I also do it as a hobby / community activity, I can say I’m pretty well educated on what the “right” answers in this area would be at any given time. For anyone who actually IS covering the actions of Microsoft full time in a professional manner, Business Applications is an area they usually openly admit to struggle in.
Power/Dataflex naming best guess as of today:
Dataflex -> Project Oakdale
Common Data Service -> Dataflex Pro -> back to Common Data Service
At the end of the day, what matters to each and every one of us the most in this MS ecosystem is how the end customers react to what they see & hear. While the marketing machine at Redmond surely is capable of capturing the attention of IT professionals worldwide with their news and campaigns, what it does not seem to be very good at is explaining “what does this really mean to you”. As the cloud platforms grow and become an ever larger pool of individual apps that form a spider web of integrations underneath the user interface, the need for someone to cut through the marketing speak and explain the customers what’s really happening here seems to be growing exponentially. What we all could do with the Power Platform is simply magical, yet the understanding of A) through which products, B) based on what technology, with C) what levels of effort and D) for what license cost – this level of understanding is lacking pretty much anywhere I look. I’m not talking about the “how to” instructions of building apps but rather the higher level discussion of where business decisions get made. It is here where the damage from confusing messaging is, in my opinion, holding this ecosystem back.
To close things off for now, as we wait for the new product names that Microsoft has said they will be giving to the technology briefly known as Dataflex, let’s recap where we are right now. “Project Oakdale” is the codename that is for the time being used to refer to the “Lite” version of CDS that will be made available to all Microsoft Teams customers at no charge. Common Data Service is what we will continue to use as the term (for now) when talking about the full capabilities of data management, entities, solutions, Model-driven apps and many other advanced features in Power Platform. This post on the Power Apps blog (originally from July 29th) is still valid in every other aspect, apart from that one “D” term that got removed on August 11th.
It’ll all work out in the end, I’m sure. This time next year there will probably be a whole number of other product names that we’ll be using when discussing the latest turn of events in the crazy & exciting world of Microsoft Business Applications. Dataflex and Oakdale will live on as one of those trick questions in the pub quizzes run by the community. No one died as a result of this episode.
In a way, us #PowerAddicts simply got a taste of what the BizApps whirwind must at times feel like for the end customer who isn’t involved with this technology of ours, all day, every day.
Microsoft dropped a big bomb this week with their Dataflex announcement. I don’t think I’ve ever had as many notifications on my social media apps as there were after I posted my two Dataflex articles on this blog and our company blog.
Knowing that it wouldn’t be a simple topic for outsiders to grasp, with the many different dimensions that Microsoft’s various product teams would use in their own messaging, I wanted to make sure there was at least one article out there that would explain what Dataflex means for Teams, Power Apps and Dynamics 365 in the big picture of business applications. It was great to see that this explanation I came up with was also adopted elsewhere in the mainstream tech media:
Here are some of the topics that the community has raised up since the July 21st announcement, based on the still fairly limited amount of public information that is out there on the coming Dataflex and the rebranded Dataflex Pro (formerly CDS).
Yeah, about that name…
For the small minority of techies who have actually heard of the Common Data Service (or XRM), the need for inventing a new name for their beloved service that remains the same (i.e. Dataflex Pro) wasn’t very obvious. For the larger crowd that works with Office tools and Microsoft Teams, Dataflex is a brand new thing, which understandably has made MS think whether a brand new brand would also be useful at this point.
Unfortunately the name “Dataflex” isn’t so unique that there wouldn’t be some clashing with non-MS products out there. In this case, there has been an existing trademark within pretty much the same application development domains since the year 1981 already.
What do you think about the following post from the owners of the #Dataflex trademark? https://t.co/WAPd9UA6Ut some people say the new #CDS name is ”Microsoft Dataflex” if I started using ”Microsoft nz365guy” I am sure I would get a letter from a lawyer. What do you think?
21 days after launching their Dataflex product name, Microsoft had to cancel their original plans due to litigation from the lawful owner of the Dataflex trademark (Data Access Worldwide). Read this blog post for more details: There never was a “Microsoft Dataflex”.
Why not call it “Power Dataflex” or “Power [anything]” then? Wouldn’t that have been better in line with the whole Power Platform story, as well as reduce the chances of a legal dispute? Probably so, but it’s important to understand that this Power thing actually isn’t the only purpose that Microsoft has in mind for the platform:
#MicrosoftDataflex it is – that is key because it applies to all of the Microsoft Cloud (that’s why no Power Dataflex :)).
The entry level offering of Dataflex for all Teams users is an important milestone for XRM/CDS/MDF (not sure I want to adopt “MDF” as the acronym just yet, actually). Don’t be surprised if the same technology will pop up in more & more places within the MS Cloud in the coming months and years.
Licensing details: TBC = “To Be Confusing”
The specifics of what is and isn’t included with Dataflex didn’t get released yet, as this new service is still pending for the public preview to start sometime in August. At the announcement day it has clearly been more important for MS to higlighlight what you can do with the non-Pro Dataflex vs. what you can’t. Even after the eventual GA, we can expect to see Microsoft use just the term “Dataflex” when referring to Pro capabilities like API access and custom code. When raising this issue of one additional source of Power Platform licensing complexity, Charles Lamanna confirmed that this is indeed intentional:
PowerApps with no spaces is just bad… I am hanging my head right now. Definitely unintentional 🙂
On Dataflex – Dataflex Pro is a licensing option (like Power BI Pro) and not actually a different thing.
This comparison to Power BI actually makes a lot of sense. In fact, much of what has happened with the platform side of MS Business Applications has been taken from the Power BI playbook used in conquering the Nr. 1 spot in business intelligence products within a relatively short period of time. So, here’s how the product levels are positioned over there:
Power BI: free for anyone to use, also for publish to web
Power BI Pro: user specific license required for sharing reports across the organization (non-public)
Power BI Premium: capacity based offering that allows enterprise customers to remove the per-user element from the licensing equation
What Dataflex and Dataflex Pro are going to offer is quite similar to this structure. No, I don’t quite expect the completely free edition of Teams to offer Dataflex features anytime soon, but the bundling into Office 365 & Microsoft 365 effectively makes it free for a huge pool of organizations to start using. When you then need to build some apps that go beyond the scenarios of team specific use cases and want to enforce some organization wide policies on it, Dataflex Pro may quickly become a requirement.
What may initially sound like not a very profitable business model (give stuff away for free) can result in a lot of positive effects on the company wide revenue streams of Microsoft if played right. After all, we’ve already seen the plumbing from under the Dynamics products to being reused as a platform for DIY apps (Power Apps), so this strategy of exposing as many users to Dataflex as possible is a clear continuation of that path.
It's the same question as "why buy Dynamics 365 Enterprise apps when I can build my own on Power Apps". Ultimately it's the scale of Teams that will bring in revenue, even if MS is (again) cannibalizing their previous offering to some extent with #MicrosoftDataflex non-Pro.
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) July 23, 2020
Following the Power BI model, the next logical step after Dataflex and Dataflex Pro would now be to offer a Dataflex Premium service for enterprise customers who are more willing to pay for the capacity rather than for each and every seat. Whether MS ever launch this or not, the question of capacity consumption in terms of storage and API calls is bound to become a big source of confusion with the “free” vs. paid plans for Power Apps. Luckily the team behind Power Platform Center of Excellence Starter Kit is already working on making some of these governance tools compatible with the non-premium resources:
We're still working on the final details, but the plan is for at least the Admin (Core) components to work in Teams with Dataflex & no need for a premium license. With the idea being you have a Teams for your Power Platform admins where they can chat, share files and use the kit
The debate on why you should not use SharePoint lists as your app’s data source vs. why they area a perectly valid option has been going on for probably longer than Power Apps has existed. The inclusion of Dataflex into the core services of O365/M365 subscriptions has now prompted people to claim that app development inside SharePoint should be avoided altogether.
There are plenty of greant points in the blog post by Andrew Welch on the impact that Dataflex will have on the app strategy for organizations. Certainly the old reality of avoiding CDS due to licensing costs will need to be replaced with a more modern view into the low-code application development landscape in Microsoft 365. SharePoint will remain as the service most well known in the Modern Workplace technology category for sure, but questioning whether it’s the right tool for managing structured data should now become a mandatory step in all app plannign discussions.
One thing that’s clearly stealing the thunder from the Dataflex announcement and making the M365 folks even more confused is the GA announcement of Microsoft Lists that took place at the very same time. Yeah, this is just classic MSFT – having multiple products with overlapping feature sets and competing marketing messages.
Teams: your business app platform?
No matter how cool the apps that we build for customers (and ourselves) may be, it’s important to keep in mind that the majority of the average information worker’s time is spent inside applications that are exactly the same for everyone – like Microsoft Teams. This recent news about Slack first claiming Teams is not even a competitor to them, then moving to filing a lawsuit against Microsoft for bundling Teams with O365/M365 subscriptions, tells a lot about the game being played:
After claiming Microsoft wasn't a competitor, Slack is now filing a competition complaint against Microsoft in the EU, claiming it is illegally tying Teams to Office 365: https://t.co/tRdoqCj6yz
It’s a fight for the future of work, which will largerly be remote work. The digital transformation of business processes will require many, many apps that are tailored for a specific task inside a particular organization. Nevertheless, it’s the umbrella above them, meaning the platform for teamwork, that will drive a lot of the technology choices and licensing dollars into the online meeting and remote work ecosystem that provides the different business apps the common context.
If there would indeed be a significant uptake on the modern task based “mini” apps that do one thing well and we’d see organizations abandond their earlier enterprise system monoliths, then the question is where would all these apps logically land in? I haven’t really seen the enterprise app store concept being heavily utilized out there in the real world yet. Nor has there been much hope for Windows to enjoy a similar success with users installing apps from Microsoft Store, like they do all the time with iOS and Android.
It’s been already a decade since the app store concept was first launched for Microsoft Business Applications. AppSource is a great channel for distributing Dynamics 365 and Power Apps solutions in theory, but in practice it’s a lousy marketplace for most ISVs to do business in:
The #MSBizApps store that's been in the making since 2010 and still hasn't been able to deliver results: Microsoft AppSource. If even @stevemordue can't get customers via this channel, it's hard to imagine any other #MSpartner can make a buck from it either. https://t.co/13QzpELulM
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) June 13, 2020
I’m not yet going to claim “this time it’s different” on the Teams app store and the Dataflex powered apps that can soon be published there by both 3rd party vendors and internal citizen developers. The idea however does sound like something that might scale better. Instead of deploying add-ons or templates into the single system of record like CRM, the new Teams based deployment model could allow a smaller group of users to install apps into an environment with a lot more narrow impact on organization-wide processes and data models. This could well be a more logical approach for Power Apps based solutions to find their way into more and more tenants.
At the start of this year, we made the decision to go all-in on Microsoft Power Platform and founded a company that focuses on helping organizations on their journey to take ownership of this low-code business application platform. You could therefore say that it’s pretty darn important for us that there is compelling roadmap for the products that represent the technical foundation of our services.
Luckily Microsoft doesn’t show any signs of slowing down the investments into Power Apps, Power Automate, Power BI, Power Virtual Agents and all the underlying elements of the platform. Our MVP team at Forward Forever already did a “Top 3 x 3” highlights article on what each of us found to be the most exciting feature announcements in the 2020 Release Wave 2 release plans that Microsoft published yesterday. I wanted to expand a bit on that top list and reflect on how I see Power Platform evolving based on this new roadmap information.
But first: what happened to Wave 1?
Ah, true. We’re actually only halfway through the April-September period that 2020 Release Wave 1 represents. This means that many of the features I highlighted in my earlier first impressions posts have not yet shipped. Just because there’s a virtual launch event every six months doesn’t mean that it would be the exact time when a big box full of new software is made available. There are no version numbers in the cloud, it’s a continuous release train that runs on its endless route.
It’s important to keep in mind that these are release plans, not release notes. The thing with plans is that they tend to change, and such is also the fate of some items in 2020 Wave 1 that I picked as the higlights in my post back in January. If we look at the change history page in the release plan, for Power Apps alone we see that 15 items had their release schedule changed and 7 were removed from Wave 1. And remember we’re only a bit over 50% through the April-September period, so more changes are bound to still occur.
Changes do happen for the better, too. 8 new features have been added for Power Apps already after the initial release plan. In total there are one hunderd new features added to 2020 Release Wave 1 after January 27th. Wow! I can’t recall how many items in total there were originally, but this really highlights two things:
The Release Plan is not a static document, rather it’s an evolving backlog of planned features.
Microsoft has been very actively maintaining the online version of the Release Plan over on docs.microsoft.com.
This is what the product roadmaps for clour services are like today. You can’t simply have a look every 6 months and then forget about them, rather you should always refer to the latest information online. Also, you can be 100% sure that there will be a lot more coming for Power Platform between October 2020 and March 2021 than this first Release Plan version reveals. Just look at Wave 1: we would not have seen from the plan that MS would acquire an RPA vendor like Softomotive, or that they would announce T-SQL support for CDS during MS Build. Get ready to be taken by surprise during Wave 2 as well!
Wave 2 features to keep an eye on
Portals Web API with CRUD support was big news in Wave 1, but the GA date for the feature isn’t until February 2021. It’s all part of the story where Portals is being made more compatible with the two other app types in the family. Wave 2 now promises a preview of PCF control support for Portals in December 2020, which should certainly be a nicely wrapped Xmas present for pro devs if that schedule will hold.
If Portals is getting closer to the mainstream Power Apps types, then the unification of Canvas and Model-driven apps into “Run One UI” is also moving along. By the time the custom pages feature hits public preview in December 2020 it will have been 18 months since we saw the roadmap, but I’m not actually that surprised it’s taking a while for making these 2 very different client technologies work in harmony. It’ll surely be worth the wait from an UX perspective, as the Canvas app embed story has been somewhat limited in its impact so far.
Once Canvas apps can be natively placed on pages that appear inside the app navigation, they’ll be ever more likely to become an integral part of more complex enterprise applications. This means that also Canvas development practices need to become more mature, which is exactly what the reusable components for business logic are aiming for. PCF components for Canvas apps are also critical to allow the pro dev audience to contribute into low-code app development projects. Code components have already been in preview for a while and GA is expected in March 2021, so obviously injecting custom code into what was originally designed as a “PowerPoint + Excel” experience for citizen developers is a big investment that takes time to polish.
Power Automate has grown into an all-encompassing process automation service that covers both API and non-API (meaning RPA) scenarios. I have to say that for me personally it’s one of the scariest parts of this “low-code” platform due to how much secret formula knowledge one must posses to achieve something where XRM workflows offered a full GUI experience. I’m glad to see that MS is investing not only in the UI flows territory from their Softomotive acquisition but also in making Power Automate work more seamlessly with Power Apps. Simplifying things even further, the coming “diet designer” and templates desined for Microsoft Teams users is an example of how broadly the PaaS foundation of Azure Logic Apps is being productized via its Power Automate UI experiences across the whole MS stack.
From the Data Platform side, we will finally be seeing the ability to use Power BI on system dashboards for Model-driven apps. It’s one of those things that the vast majority of customers probably would have assumed to be possible for ages already. Getting proper support for parameters like environment variables has been a prerequisite to make these components play well together in the Power Platform ALM story. Now, this still doesn’t mean PBI would replace the built-in visualizations from CRM 2011 era, rather we see that MS is actually only working on catching up with the year 2011 when it comes to Unified Interface charts customization (i.e. supporting ASP.Net chart XML based features with the new Highcharts).
Come to think of it, there isn’t a single mention of the TDS endpoint / T-SQL support in either 2020 Wave 1 or 2020 Wave 2 release plans. It’s a good reminder to everyone that despite of the huge volume of information in the release plans, they don’t reveal the complete story of what’s happening with Business Applications beyond the immediately visible end user and app maker features. You’ll still need to do a bit of 1+1 yourself to figure out how things like DirectQuery support enabled by the TDS endpoint might have a dependency on unlocking more modern visualizations on top of CDS data in Power Apps.
While creating charts on a sales pipeline etc. could be seen as just doing old stuff with new tech, what’s really net new in terms of data analysis capabilties is all the goodies coming into CDS integration with Azure Data Lake. Time series data: get the full historical values of business record in a format that you can actually report on (i.e. not audit logs). Soft delete support: instead of keeping all historical data in that fairly expensive CDS database storage, delete the transactional record but still keep the data in the analytical system. Support for entities with attachments: if you’d like to get some value out of the annotation data clogging up your CRM system, push it into the Data Lake where AI can crunch it and generate new insights from it. All in all, this built-in continuous replication of data from the Power Apps / Dynamics 365 app database into Azure Data Lake seems to be truly delivering on the vision of The Real Common Data Service that goes far beyond the boundaries XRM used to have.
If getting data out of CDS is evolving, so are the mechanisms for getting data in. Power Platform dataflows have a lot of potential for sure, yet just like with Power Automate, it’s been difficult for these new generic cloud services to complete with the built-in XRM tools when it comes to feature completeness for CRM customers to achieve their common business goals (use a GUI to automate a process, run a Wizard to import relational data). Power Automate already has the path to “upgrade” the business logic into Azure Logic Apps and now also Power Platform dataflows can grow up to Azure Data Factory, once the March 2021 preview arrives. In the big picture, this ability to start from citizen developer tools and then transition to pro dev methods and systems is a very powerful value proposition from Microsoft to all their customer organizations who are looking to empower their subject matter experts to take steps forward in the day-to-day digital transformation. App creation & automation is where this Maker revolution has started, and I bet we’ll see more and more data driven features in the next phase of Power Platform evolution.
Finally, as the number of apps grows and they become an irreplaceable part of critical business processes, there’s a growing demand for tools to ensure the digital machine is well oiled. As we know, data is the new oil and in this context that means organizations need broader access to telemetry data on how their business apps are working. The 2020 Wave 2 feature that promises to deliver CDS errors, performance data, and diagnostics data in customer’s own Azure Application Insights should do exactly this, by complemeting the built-in metrics available in PPAC with a way for customers to build their own alert systems for both proactive and reactive maintenance. Yet another “grow up to Azure” path that illustrates how closely the low-code application platform-as-a-service will be intertwined with Microsoft’s PaaS offering.
Application Lifecycle Management, ALM, is a topic rarely covered in the mainstream pitch for how low-code/no-code solutions are transforming app development practices. This is understandable, since the initial time-to-value, from an identified business need to a working application (or at least a functioning prototype of one) is where the major difference to the traditional software development initiatives are easiest to demonstrate. It’s the “five minutes to WOW” effect that gets most of the attention.
Relying on configuration and formulas over custom code does promise also a lower maintenance overhead for the post go-live phase where the apps are used and evolved gradually to remain in line with the business requirements. There is less to worry about the potential impact of changes in each of the underlying components that keep the app running when you’re subscribing to a service like Power Platform that aims to abstract away many of the moving pieces in Azure and other related cloud services.
You shouldn’t assume life to be completely worry free, though. If you don’t plan ahead on what type of work is needed after the creation of your low-code app, then a nasty surprise may lie ahead for someone, somewhere, at the most inconvenient time. In the worst case your organization has become reliant on an app that no one owns & no one knows how it works, only to realize that your processes have already stopped due to errors caused by a change either inside or outside the system. Updates to the cloud components or changes in APIs used via Connectors, for example. A slower yet equally frustrating problem can arise from identifying a future change you’d really need to implement for the app in question, yet you don’t have the means to make that happen in practice.
The rise of low-code application development and the forecasted arrival of 500 million new apps in the next 5 years is making IT organizations very aware of the need for setting up governance models to address this new reality. Having baseline knowledge of what apps are created, by whom, for which audience, for what purpose, through what technologies, on what data sources etc. is often the discussion that needs to be done first to analyze the current situation. A great tool for facilitating this is the Power Platform Center of Excellence (CoE) Starter Kit, which gives you an overview of the big picture for Power Apps and Flows in your tenant. To go deeper and actually manage the lifecycle of an individual (potentially business critical) application, we need to start discussing ALM.
What is this ALM thing anyway?
Microsoft has published fresh new documentation on ALM with Power Platform that could be viewed as a similar Starter Kit for this topic as the aforementioned CoE. It briefly covers the high level summary of what ALM means:
ALM is the lifecycle management of applications, which includes governance, development, and maintenance. Moreover, it includes these disciplines: requirements management, software architecture, development, testing, maintenance, change management, continuous integration, project management, deployment, and release management.
A very broad concept then. What I like most about this intro chapter is the definition of the goal of ALM: “driving efficiency through predictable and repeatable software delivery”. That’s the essence right there. Anything that reduces the differences between how individual app makers / developers perform common tasks within the whole lifecycle of a specific business application will improve the state of your ALM practices. Those with a software development background may immediately venture into thinking about various DevOps methods and tools at this point, whereas I’m always most concerned about how the business can get a grip of what exactly is happening to their applications over time. Who built what, based on which specification, when was it deployed & where to, how were the changes documented, what impact did it have on others systems, how were users educated on the new features, and so on.
Microsoft’s ALM guidance is mostly revolving around the technical level specific to Power Platform, meaning how solutions and environments should be used. If you haven’t heard it before, then the solution framework that was born in 2010 for the XRM era has been actively expanded to cover more & more areas that are not a part of the Dynamics 365 Customer Engagement on-premises server. Canvas apps, Flows, AI Builder models, Power Virtual Agent chatbos, Export to Data Lake configs, pretty much everything in the Power Platform is now manifested as solution component. More is on the way, based on how Matt Barbour has stated things like Power BI and integration services being “on the radar” for the Power Platform solution system.
The expanding technical capabilities and scope of components shipped via solutions is of course great news. However, as we are now talking about the platform for every developer, not just pro devs, my number one question about Power Platform ALM is not “will it scale up to new products and features” but rather “will it scale down to new types of apps that don’t follow the lifecycle of a CRM system?” So, there’s two dimensions I feel will be critical for the ALM story to succeed: 1) unifying the mechanisms to manage configuration & code across the different app teams within MS Business Applications Group, and 2) democratizing healthy ALM practices by making them accessible to citizen developers. The first one is well underway with the solutionization of everything, but how about the second one?
Managing your “source low-code”
The fundamental challenge with why ALM tools aren’t already a part of every Power Apps maker’s or Dynamics 365 customizer’s core workflow is that they don’t exist within the context where these people do their work. This audience doesn’t need Visual Studio for anything, nor would there be anything familiar with the terminology used by version control systems like Git. This is alien technology from Planet Developer, which the no-code app makers would have to learn specifically for the purpose of managing how their configuration deliverables are moved from one environment to another. Compare that to the “export solution as .zip” / “import solution .zip” actions availble in the Power Apps Maker GUI that gets the job done for them. Many of these app makers surely understand the value that a formal deployment process and solution versioning could offer, but the barrier has been much too high for them to make that leap.
There have been tools for pro-code developers working with XRM solutions for many years already, some provided by MS and many built by the community. Today there are official Microsoft Power Platform Build Tools available for Azure DevOps, which offer readymade tasks like solution export, import, unpack, pack, publish, with more features to come. Rather than tweaking PowerShell scripts, it would seem like there’s a way to visually configure the cloud to do some ALM magic for our low-code apps. Sure, this is not within Power Apps directly, but could this be considered just another admin portal that the citizen developers might learn to navigate?
Setting up an Azure DevOps pipeline that pulls solutions from your dev environment, commits them into a repository for version control and then deploys that same solution to test or prod still isn’t exactly a next-next-next process. However, following a Power Platform specific tutorial, even a non-developer like me can get this up & running in a few hours. MVP Nick Doelman has written exactly what I needed, so if you’re interested in building your very first ALM pipeline, I recommend you to try this out:
Combining this with a recent update to the Power Apps Build Tools that also introduced support for MFA protected environments (meaning you can log in with an Application User identity), I was able to get our company’s “CRM” solution’s customization changes from from Dev to GitHub and Dev to Prod. Here’s what it looks like now when adding a new field on the accout entity:
“Woo-hoo, we now have proper ALM in place!” Well, not quite. All that we really achieved with this was the possiblity of doing it “right”, meaning there are DevOps pipelines that could be used for handling configuration changes automatically instead of manually by individual low-code developers. Until I actually go and remove all Power Platform service admin roles from my colleagues, anyone of them could still go and mess up the production customizations directly. So, proper ALM is only partially about the tooling and automation – you still need to the exact same thing as without DevOps, meaning defining the processes and agreeing on the practices for how the business applications are managed.
Regardless of this, it does makes perfect sense for low-code app development projects to evaluate if they would get benefits from leveraging automation tools like Azure DevOps. You’ll need to have the solution configuration stored somewhere anyway, so why not put it in a version control system that’s designed for “real code” and gain access to things like viewing the changes of individual solution components – rather than just dumping the files on OneDrive/SharePoint. Who knows, taking that step might even unlock further opportunities to automate and harmonize your application management processes.
We should keep in mind that automation is a double-edged sword. In theory it can save you a lot of time, yet in practice it may often turn into an additional time sink of its own. The legendary webcomic XKCD beautifully illustrated this dilemma of Automation, which I think is especially true here in the ALM discussion. The tasks that you’re trying to automate need to happen on a frequent enough basis for a long enough time before you could achieve a positive ROI from investing in setting up your pipelines. A single citizen developer building a one-off app may not yet meet that threshold, so preferably in your automation scenario there should be a team of developers who will do several planned releases for the app in question.
ALM for a team of low-code developers
Microsoft likes to promote the stories of Power Apps Champions who have managed to make digital transformation a reality for their organizations by building low-code apps that would have earlier required a team of software developers. This marketing approach higlights the role of an individual hero, but from an ALM perspective the dependency to any individual person’s heroic actions and secret knowledge is rather a problem that should be solved. Once that initial app idea and first PoC starts turning into something business critical, you should have a team in place that can ensure its continuity.
This team may not resemble your typical software development team. Yes, it’s quite likely that there will also be people with programming skills either working in the team or closely with it, but the primary competence on the technology side for this team is in assembling the features and components of the chose application platform into solutions that solve a wide variety of business problems. They are less likely to be all working with the single code base of a large application, doing pull requests, commits and all that Git jazz, simply because a fair share of Power Apps aren’t going to be built that way at all.
If you look at Power Apps Canvas apps today, they are essentially a document made of JSON that cannot be simultaneously worked on by multiple developers. This is very different from a traditional app project on the Model-driven side (let’s say in a CRM context) where one person could be building the customer data management features while another team member is working on products and order management. All with their own dedicated entities, having so little overlap that simultaneous work within the same dev environment is often doable. Even though Components promise to offer a more modular way to build capabilities for the Canvas world (and Model, too), I suspect that the client side experience of a single app will typically remain a “one maker show” in the near future.
Sharing elements and patterns across different apps built and managed by the team is, on the other hand, in a much greater role in the low-code approach. Here components, template apps and other reusable pieces of configuration can be of high value. Power Apps began its journey as a citizen-first app creation experience yet is gradually accumulating more of these features that mean every app doesn’t have to be a unique project. We’re getting more & more possibilities to see beyond the Maker GUI, like “peek code” in a Flow or use Monitor to debug Canvas app formulas. However, based on what Microsoft representatives have publicly said, it doesn’t sound likely that these low-code tools would evolve into allowing app makers to directly manipulate the configuration files in VS Code, for example. There are community tools like CanvasAppPackager from MVP Daryl LaBar that will help app developers understand complex Canvas apps, but these are NOT meant for building or modifying apps.
Even in Model-driven apps the list of supported modifications for customizations.xml is fairly limited. So, keep this in mind when thinking if the pro-dev tools and ALM practices that are based on direct source code manupulation are going to be a fit for the low-code app developer team or not. Your Power Platform experts need to be primarily fluent with the admin tools from MS and the community (always start from XrmToolBox), plus they’ll need excellent skills for using collaborative tools like Teams within your org to share their lessons learned. Once you’ve got this side is covered, then you can move onto Azure DevOps and the likes.
Environment management is another big topic for low-code apps on Power Platform. Having a separate CDS environment for every developer working on their corner of an application is Microsoft’s recommended approach to keep changes isolated. As your number of “ALM enabled” apps in the tenant will grow, the total number of environments will grow even faster. Assuming a theoretical 5 GB size for the CDS database of an environment, the $200/month storage cost itself wouldn’t actually be that significant if it saves a few hours of work from those developers. What you don’t want to happen, though, is having dormant dev environments consuming CDS capacity for many months with no actual deliverables from them – especially when your CDS quota is defined on a tenant level with no control available for reserving it to specific apps/environments.
There are remedies to this problem. The “governance way” is managing the environment creation/deletion though a process that provides transparency to environment usage and enables allocating costs accordingly, for example via custom apps built on top of the CoE Starter Kit. The “ALM way” is provisioning on-demand environments from the source code in your repo via DevOps automation, then scrapping them after the active development work stops. While I sort of fancy this idea of “everything as code” where all of the infrastructure and configuration is managed via the same pipeline, I can’t help thinking that the most difficult thing about environments for business app development isn’t necessarily the configuration or the code. It’s the valid test data needed for making the app come alive. Yes, of course you could also script the perfect test data set to be generated alongside the environments, but again, we’re taking the level of effort needed in achieving deployment automation onto a very non-citizen developer level and approaching the XKCD scenario.
Managed Solutions: what do you REALLY want to manage?
Whenever Microsoft talks about ALM, usually you’re going to hear their statement on “use only managed solutions in production” sooner later than later. It’s not surprising that also the latest guidance, such as the MBAS 2o20 sesssion on Advanced Application Lifecycle Management capabilities lists this high up on the ALM Health Check list:
In the user community around XRM there’s been an endless debate around whether unmanaged or managed solutions are really the way to go, and we’re not going to find the ultimate answer to it here in my blog. Personally, I’ve always had a hard time understanding the exact benefit that a customer organization would get from self-imposing the limitations of managed solutions onto themselves. It just sounds to me like installing a different lock with a dedicated key onto the window blinds in every room of your house. Expensive, overly complex, and very likely to leave you in the dark when someone forgets which key was used for which lock. It can also give you a false sense of security, as the main door to your house is still left completely unlocked, because Microsoft doesn’t yet offer a way to protect your production environments from the deployment of unmanaged customizations by anyone with sysadmin rights.
In the context of low-code specifically, if you don’t have a seriously mature ALM process in place, I’d say you’re fairly likely to hurt yourself when playing with the sharp objects that are managed solutions. This is why during my 10 years of working with Dynamics CRM, XRM and now Power Apps, I haven’t yet deployed a single managed solution inside a customer specific project. On the other hand, I have worked with removing managed solutions created by the customers’ previous partners, after the ALM process has broken down for one reason or another (“not existing” being one of them), then replacing them with an unmanaged layer of customizations that has been a better fit for their overall CRM landscape. Yes, those other partners probably defaulted to following MS instructions on using managed solutions in every production environment – this guidance remained the same since CRM 2011 days, after all.
Please do note that I’m not saying managed solutions wouldn’t serve a purpose. When you are building an actual app product that is meant to be deployed across multiple environments for different customers, managed solutions are of course the way to go. They offer the container for keeping your stuff separate from the stuff delivered by other app vendors, when deployed into the single CRM system that’s used widely across the enterprise by different user groups for different processes. Any proper commercial product development initiative should also be able to absorb the ALM overhead from environments, automation etc. and turn that into increased revenue from being able to ship software releases faster and more reliably as time goes by, thanks to the upfront investment made. It all makes sense here, but this is a very different landscape than how I perceive the typical use cases for low-code apps within organizations to evolve as Power Platform becomes more widely adopted.
Remember: you’re no longer building “one place for all customer data”, rather you’re often creating several targeted apps for very specific purposes and audiences. Most of these app experiences will likely be fully developed by either savvy citizen developers, an in-house team of Power Apps champions, or an external consultant brought in to get that job done. The app lifecycle can be very short, even targeted to be used in one-off events (or under temporary disruptions such as COVID-19 social distancing), whereby some apps may actually be “disposable by design”. These are pretty much the opposite of monolithic CRM systems where a wealth of different ISV solutions need to peacefully co-exist throughout the years. So, whichever solution strategy you choose for your Power Platform environments, make sure it solves a real problem that your low-code app developers and admins are facing, rather than directly adapting practices that have been designed for the ALM needs of pro-developer teams working full-time on shipping a sizeable amount of custom code alongside the low-code application platform configuration information.
A different approach to ALM: Power BI deployment pipelines
The broad umbrella of Microsoft Power Platform covers also technology that isn’t (currently) built on the solution framework: Power BI. Since it is in practical terms more part of MS Data Platform, the mechanisms used on the BI side are understandably quite different than those originating from the citizen-dev PowerApps (back when it was still spelled together) or CRM systems. What makes this an interesting area to observe is that from what I’ve understood, the traditional development of reporting solutions hasn’t exactly followed the typical software development path either. Checking in reports to source control systems may not be the natural workflow of a Power BI content author or data analyst, so the barrier for adopting advanced ALM practices is likely to be pretty high for this audience, too.
Power BI deployment pipelines is a feature that has recently been launched in preview. Labelled as an ALM solution, as also Power BI now packages report content into “apps”, this functionality is actually baked right in to the native user interface of the PBI service:
Reading the introductory text gives us an idea of how this feature is positioned:
Deployment pipelines help enterprise BI teams build an efficient and reusable process by maintaining development, test, and production environments. BI creators can incrementally transition new or updated content between environments, reconfiguring them with the appropriate data connections and permissions. Pipelines are very easy to use and take only few minutes to setup. No previous expertise or coding skills required.
Wouldn’t it be nice if we could take the term “BI” from that paragraph, replace it with “App” and have a similar story to tell for low-code business app developers? A targeted experience that would be readily available for each & every customer environment right out of the box would certainly be the most effective way to democratize healthy ALM practices.
You’ll need Power BI Premium capacity for using deployment pipelines, so the reporting ALM story is aimed towards the enterprise audience. If there ever was a native deployment pipeline functionality for Power Apps, I’m sure the usage would require consuming some type of paid for capacity from the Power Platform product subscriptions. I believe this would still be a lot easier approach for selling the idea of why customers need to invest in having proper ALM infrastructure and capabilities in place, since it wouldn’t be just an extra layer of billable hours added on top of app development project budgets but rather something much more tangible.
Last week I came across an article on LinkedIn, where someone was saying “the time for Social CRM is upon us, thanks to COVID-19 changing the customers’ behaviour, and here’s a CRM system that you should use for managing it” (I’m paraphrasing). What caught my eye in the text was the references to the writings of Paul Greenberg, the semi-official godfather of CRM, in the 4th (and last) edition of his bible “CRM at the Speed of Light”. Released back in late 2009, the 700 page worth of content are probablt too much for anyone but the most hardcore CRM fans to consume at this point, but Paul’s blog post “Time to Put A Stake in The Ground on Social CRM” certainly is worth revisiting even in 2020.
Of course those who have followed Paul’s writings over the years may remember that he made the decision in 2013 to drop the “Social” from Social CRM and just call it CRM once again. The reasoning behind it was just as sound as was the original text laying out the definition of SCRM. I want to emphasize that I don’t consider the era of Social CRM from 2008 to 2013 to be a mistake of any sort. What I did find a bit awkward is the idea of selling any suite of software as the platform for “Social CRM” in 2020, since what Paul Greenberg wrote already back in 2009 for his Stake in the Ground article is still very much valid:
“When you look at the SCRM applications out there – there are no actual SCRM suites, no matter what the claims of any company on either the CRM or social tools side. What you do have are effective and important applications that increase the ability of employees to interact with customers – though they are not tools that facilitate the actual interaction. You also have the integration of social media and community building tools with traditional CRM tools which are providing effective combinations which are leading toward SCRM. I want to emphasize. These are all good tools. They are worthy of any company’s consideration. There is just no SCRM suite out there – as of yet or in the near future. Which doesn’t matter one iota.”
In the same vein as you can’t simply “do CRM” by just deploying a CRM software suite while skipping the strategy part and business process implementation, Social CRM was never about waiting for a version of your CRM suite to arrive that included the missing social module. This is not a radical piece of insight by any means but rather common sense – something that needs to be stated also on a technical blog like mine every now & then.
What actually inspired me to start writing this article about Social CRM is the image that I saw in my Timehop stream yesterday morning. For those who don’t know, Timehop is a service that creates a bucket of digital memories from the same date X years ago and delivers it as a push notification on your smartphone every day (I wish the still did the email digest but that’s another rant). 10 years ago I was attending the Microsoft Convergence conference in Atlanta and attending a session called “Deriving Value from Social Networks”, delivered by Nikhil Hasija from MS and Paul Greenberg from… well, you know Paul already. The picture I had grabbed from the session was this #Customer #SCRM Stack slide:
Not just that, I actually wrote a review about the session in my blog that used to be called Surviving CRM. With the whole Social CRM topic fresh in my mind I needed to go and revisit my notes from a decade ago, to gain some perspective on what the world looked like back then and what I was expecting to see from technology vendors like Microsoft on this social front.
The early 2010s were a time of Social CRM technology acquisition spree. If you want proof, the company that published this infograph on the Social CRM Arms Race between Oracle and Salesforce in June 2012 (which I referenced in my later blog posts) had actually found itself to be part of Salesforce only one year later – after having first been acquired by ExactTarget. Microsoft was late to the game and playing with a considerably lower stack of chips than these two. In hindsight, it may have been wise to not pour in all of their money into this one table, since it’s hard to say if the ROI has truly been there for any of the large tech corporations that were chasing the elusive Social CRM suite at the time. Still, MS naturally had to take part in the game, which is depicted in this 2012 slide on how the Dynamics team perceived the role of Social in their product strategy:
Viewed through the lenses of a MSFT ecosytem player like myself, here are some of my own observations on how their strategy eventually played out in different areas of Social CRM.
Social network data streams
The big data from public social feeds like Twitter and Facebook was at one time considered to be so critical for your SCRM product offering that Microsoft went out and bought a company called Netbreeze in 2013 that had built a system to harvest social posts and produce nice looking analytics on top of it. Initially named as Microsoft Social Listening and then Microsoft Social Engagement (MSE), the service was offered to basically all Dynamics CRM Online customers for free. This unfortunately meant most of them didn’t actually perceived it as a serious business tool, which I’d imagine also affected the product development budget in a negative way, leading to a vicious cycle.
In the beginning of 2019 Microsoft announced it was going to shut down MSE, and at the same time launched a preview of a new service called Dynamics 365 Market Insights. Then a year later also Market Insights was discontinued, with the message that some of the capabilities will find new homes in Dynamics 365 Customer Insights or Microsoft Bing industry updates. One thing to keep in mind is that the actual social data streams of Twitter and Facebook had already been removed from the scope of the product before that, so the statement doesn’t necessarily cover those ever making a return.
After making investments in both acquisitions as well as product development over the past 8 years, Microsoft has now effectively made an exit from the business of social listening and social engagement. Partner solutions from the likes of Sprinklr, Orlo or Hootsuite are now offered as the systems to use for social media management with their integrations to Dynamics 365 apps. Entities like Social Profile and Social Activity remain in the Common Data Model schema, but I’m not sure if there’s any service feeding data into them after MSE is gone.
I don’t believe for a moment that Microsoft would not be seeing value in the data itself, rather I think they are aligning themselves behind a much more realistic strategy now. Building upon the strength of Azure data platfrom, Microsoft’s Customer Data Platform is in a position to make them a leader in CDP rather than a follower in social data processing. This is much better in line with how Satya Nadella has been transforming MS from their early OS monopoly roots into a team player for the cloud, seeking to create value via new connections in the global network of people and technology, rather than trying to (immediately) bring down their competitors.
Social selling
While there is far more data in the social media firehose of Facebook, Instagram, Twitter etc., the one source that everyone in the B2B business of CRM have always had their eyes on is LinkedIn. It is their API that was the foundation of so many early Social CRM tools like Nimble back in the 2010 era. LinkedIn’s decision to lock out these smaller CRM providers in their 2014 “LinkedOut” API restrictions put an end to the era where building a sales app of your own on top of a database from someone else had seemed like a good idea.
Undoubtedly the biggest move that Microsoft has ever made in the social field was the 2016 acuisition of LinkedIn for $26 billion (note that this was already during the Nadella reign). As this deal was reportedly a competition between Salesforce and MS, it was only natural that the initial assumptions were for this new data source to light up in the Dynamics products and change the game of B2B CRM forever.
It hasn’t happened yet. What we’ve seen so far on the Dynamics 365 side has been mostly just the Microsoft Relationship Sales offering that embeds content from LinkedIn into the UI of Dynamics. Sure, there’s a bit more to it on the functional side, but it still closelt resembles the LinkedIn Sales Navigator that Paul Greenberg has always hated, calling it “a faux middleware to get access to the LinkedIn data”. In the standard Dynamics 365 Sales Enterprise app, there still isn’t even a LinkedIn profile card available yet for contacts, whereas Outlook on the web shows one. All in all we’re still surprisingly close to what the social selling story for Dynamics CRM was back in 2013.
Instead of using LinkedIn as a weapon in the battle for CRM supremacy, Microsoft appears to have chosen to play the long game – and also to play it safe. I belive the major driver behind this approach has been the hightened attention on privacy. Back in 2010, there wasn’t a widespread awareness yet on the potential privacy issues that the global social networks have introduced into our lives. Today pretty much everyone has heard what the NSA had been doing with their social data archives, as well as how Cambridge Analytica gamed the election systems with the help of Facebook social profile/network data and ad targeting. We’ve seen GDPR come into effect in the EU, forcing companies from around the world to pay attention to their policies and procedures around customer data handling.
The real strategic value from LinkedIn isn’t in the straightforward enablement of social selling via offering data to users of a CRM system, rather it is in the “professional graph” that Microsoft can build when combining this with signals from its own services like Office 365. Yes, in the end it will help both Microsoft and its customers to sell more products and services in the digital world, of course. It’s just going to be more AI driven than the original visions from a decade ago, with MS keeping the network data closer to itself and the faceless algorithms that are crunching the big data to deliver smart recommendations to the app users.
Social customer service
As mentioned, acquisitions were a popular path for CRM giants to build up capabilities to match the new social customer era in the early 2010s. Customer service was an area where Microsoft originally attempted to bring the self-service capabilities of online knowledge bases and Facebook page embeds into their CRM suite by acquiring a company called Parature. We hardly even saw that product in the European market before it was discontinued, in favor of a long & winding internal feature development path to build a foundation on top of which the customer organizations could build their service processes (often still requiring plenty of custom code). While SaaS products like Zendesk weren’t necessarily customizable enough for enterprise needs, what they did was show a level of modern customer service experiences that seemed very difficult to reach when building up from a traditional CRM system design to be used behind corporate firewalls.
That’s ultimately what was always weighing down a product like MS CRM in the Social CRM race: it was designed to be an internal system. Exposing the data to the outside world meant that you had to re-think the system architecture in a major way. Sure, there were visionaries already back in Convergence 2010 like Adxstudio that powered the portal accelerators for CRM Online. Later MS decided to acquire the company and today that technology also powers the Power Apps Portals aimed at citizen developers, to quickly build self-service interfaces to access line of business data via responsive web interfaces.
Omnichannel has been a big investment area for Dynamics 365 in the past few years. While still working on their own web chat, co-browsing and call/video technology for the public web, Microsoft partnered with CafeX to offer a stop-gap solution to allow live online support experiences on their Portals. The recent updates to Omnichannel are now finally bringing also non-MS channels into play, with support for WhatsApp discussions (via Twilio), Twitter, WeChat and LINE. The modern Azure-based architecture of Microsoft Teams provides a solid foundation for future expansion of messaging functionality. You could say that after a slow start and a few side steps, the social customer service story has developed into a promising direction.
Social events
The backchannel of physical events that takes place in social media, under specific hashtags, was a big new phenomenon back in 2010. It was around that time when I started to actively follow and curate content from conferences around the world, from the comfort of my own home, armed with just a set of Twitter streams on my Tweetdeck client. Actual live streams of the sessions were still very rare, but the community around MS Business Applications strated to come together precisely because of the key role of individuals in spreading the word on what was being presented on the live stage.
As it just happens, 2010 was the year of the ash cloud, which stopped a significant slice of all air traffic globally and forced conference attendees like me to look for alternative routes to get to the venue from the other side of the globe:
10 years ago flights were grounded only on one side of the planet, due to the ash cloud. Compared to #COVID19, those were actually pretty fun times, as there was a way around it all. Today we're grounded indefinitely and there's nothing anyone can do about it😕 pic.twitter.com/vW516wrrWr
— Jukka Niiranen mstdn.social/@jukkan (@jukkan) April 17, 2020
Looking back at what a single volcano did in 2010, it was a tiny little blip compared to the COVID19 pandemic we are facing right now. In trying to adjust to the new reality of a world full of coronavirus, Microsoft has announced to go digital-only with its own events for at least the end of June 2021. While I belive this is the right move for them to make, I also belive Microsoft will take this as an opportunity to build up their own technical capabilities for running virtual events and offering social engagement channels around them. This would surely be a very welcome direction for Microsoft’s customers, since these capabilities haven’t progressed as much from the year 2010 as you could have expected. Let’s look at a few key elements that could use bigger investments:
Webinars. Those who participate in the MS partner ecosystem events may well have run into the ON24 webinar platform. It also happens to be the only supported webinar provider for Dynamics 365 Marketing today. To give you an example of the event attendee experience for ON24, let’s just say that the little red icon you see in the Slides window below belongs back to 2010, not this current decade that the rest of us are living in…
Event recordings. Microsoft does run a service called Stream that’s included in Office 365 (Microsoft 365) subscriptions, but it doesn’t allow sharing videos with anyone outside of your tenant. Not even guest accounts that can sign into the Microsoft Teams teams are allowed to view event recordings from Stream. Since this limitation has been around for over two years already, I suspect we’re not very close yet on Stream becoming a real competitor for the public video sharing services like YouTube or Vimeo and their community features.
Slides. Together with their LinkedIn acquistion, Microsoft became the owner of SlideShare. While initially it would seem like an obvious “better together” story to have the makers of PowerPoint united with the leading social platform for sharing slides, this has obviously not been an area that MS has prioritized. In fact, the SlideShare service seems to have been abandoned years ago already. Instead of developing new features, even old useful ones are now being stripped away. You can no longer sign up for a company account, as this option has been silently removed, so our new company had to give up on the idea of distributing our webinar slides on a MS hosted service and instead just publish them as PDFs on our WordPress site.
Social apps ecosystem
The Social CRM movement was all about putting the customer in control of the conversation with the company, thus leading to an inevitable growth in the number of different services and digital touchpoints where the interaction would take place. Therefore it was no longer possible for companies to build a “one perfect system” to govern all the relevant processes and messages, rather the CRM systems had to become more compatibile with the outside world and the various services where the conversation took place. In practice you needed to support an ecosystem of apps and service providers to partner with one another and to offer connectors between the data sources and targets.
The technical infrastructure in 2010 wasn’t ready for this at all yet. Back in Convergence 2010 it was the first major event where the upcoming solution framework in Dynamics CRM 2011 was being presented. This mechanism was going to allow apps from different providers to be plugged on top of the common platform (and also unplugged) without the work traditionally required from a systems integrator that managed the CRM system. The CRM Online service launched globally on the following year would also heavily depend on this new modular model of extending the CRM capabilities with integrations to new cloud services without disrupting the core business processes that were relying on this system of record.
In 2010 the Apple App Store had already been around for a couple of years and this model was envisioned to become the new standard for also how business software capabilities were distributed in the near future. Out of all pieces in the puzzle for Social CRM technology, the inability to build a viable marketplace for ISVs from also outside the immediate MS ecosystem was perhaps the biggeset stumbling block for Microsoft. What was launched as Dynamics Marketplace in 2010 became pretty much all that there would ever be: a random listing of solutions and service providers that did very little to help customers in discovering the right building blocks for the latest social technologies and services for their CRM systems.
This part of the journey has been incredibly long, but on the technical side of things we’re still moving along. The solution framework that was being previewed for XRM in 2010 is being developed to support the whole range of new Power Platform services in 2020 that now share the same foundation. It’s pretty wild that the low-code tools being offered to citizen developers across all corners of organizations are now riding along in the same freight train that originally consisted of only the sales, marketing and services cars of CRM.
This is representative of a larger movement that I think has also changed how CRM and social channels are coming together these days. Moving most of the core CRM systems into the cloud for modern organizations has made them far more accessible to social data, even without tightly coupled integrations via packaged apps. Technologies like Azure Logic Apps and Power Automate (Flow) have lowered the barrier for connecting systems together and thus made the sales, marketing and service departments a lot more agile in adopting and integrating with new social tools. In a way, the need for an SCRM suite to be built by a big tech corporation has somewhat diminished as a result of this, because customer organizations are now better equipped in assembling their own unique toolkits for engaging in social channels while connecting to their systems of record.
“One more thing”
Ah, nothing takes you back in time to the days of early Social CRM like that iconic phrase from a certain tech company’s keynotes… Anyway, to close off this exploration into the days gone by, here’s one more gem that Timehop delivered to me from 10 years ago:
For once I can say “I told you so”! Now, this particular tweet was surely about providing the customers to the option to change the visual branding of their system, but in retrospect it almost looks like if Microsoft themselves wanted to break free from the cages of the CRM acronym already way back then. Splitting the Dynamics 365 suite into separate apps and stopping the use of the term CRM was a controversial move back in 2016, which appears to have only accelerated the juggling of product brand names at MS in the past few years.
The convenience of having an established acronym to describe what you’re doing is definitely very tempting for us lazy human beings, and attaching terms like “Social” into it must seem like the easiest way to explain something new while still holding on to all the old stuff. In the end, though, Social CRM became the everyday reality through routes we might not have expected to travel – and we stopped calling it both “Social” and – on the technical platform side – “CRM”. Or did we?
On March 27 2020, Microsoft announced that the Dual-write feature is generally available. After being in preview for over a year, the functionality itself isn’t necessarily a surprise to anyone who’s actively following what’s taking place in the Dynamics 365 product portfolio. The GA does however mark a milestone where it makes sense to reflect on what the broader impact of Dual-write is to business applications built on top of Microsoft Power Platform.
The very short answer to “what is Dual-write” would be that it’s a built-in feature for making business data available across Dynamics CRM & ERP products without any additional integration tools. It may come as a surprise to people outside the MS ecosystem that even though the Dynamics 365 brand was announced already in the summer of 2016, the practical work of bringing the different apps from this product suite together has not yet changed that much from the early days of custom integration projects. Putting things into context, the effort to unify the 1 CRM & N ERP products in Microsoft’s portfolio originally targeted its first deliverables in 2004 (as “Project Green”). This proved to be impossible (or rather not feasible from a business perspective) in the on-premises era, but now when the Dynamics products are services hosted in the global MS cloud, the unification has actually been doable.
Dual-write isn’t your everyday CRM-ERP integration, though. It is a near real-time synchronization of data between the Finance and Operations database and the Common Data Service that powers the low-code application and automation development story of Power Platform. Looking at the even broader data story that Microsoft is building around it’s business apps, BI and AI capabilities, we can also see that Dual-write is just one of the mechanisms that will move business data around:
Taken from the Ignite 2019 session “Common Data Service with Dynamics 365 Finance and Operations”, this slide highlights the role of Azure Data Lake Gen2 in the analytical features that will be offered both as preconfigured features from Microsoft as well as customer specific implementations, to derive new insights from the business data. When it comes to the more operational side of doing business in the digital world, having the SCM and Finance app data available in CDS is what opens up plenty of new interesting scenarios. What if anyone could easily build Power Apps that work directly with the enterprise master data from ERP like customers, products and pricing information? If CDS indeed could lower this data integration barrier and bring task based experiences with pixel-perfect Canvas apps into the hands of frontline workers, that could surely be a transformative service worthy of its name. Not just a single database like XRM was but a connecting layer across many Microsoft cloud technologies.
In order to bring the ERP data into CDS, Microsoft needed to design harmonized data models that align with what used to be the independent products of Dynamics CRM and Dynamics AX originally and define how the schema of each system maps to the other. As an example, the CRM product data model has understandably been considerably more limited than that of an actual ERP application. Now with Dual-write, these two must come together in a unified product experience:
No two businesses operate exactly alike when it comes to data and processes, so the logic of Dual-write has been built to allow no-code configuration to adjust the details of specific implementations. However, this new depth of application integration within the Dynamics 365 portfolio now means that you should have very good reasons if you decide to deviate from the standard model. Let’s look at Microsoft’s guidance for the above mentioned product data model as an example:
The dual-write entity maps for products have been designed to flow data one-way only, in near-real time from Finance and Operations apps to Common Data Service. However, the product infrastructure has been made open to make it bi-directional if required. Although you can customize it, it’s at your own risk, as Microsoft does not recommend this approach.
Anyone who’s worked with Dynamics CRM / Dynamics 365 Customer Engagement implementations in the past has surely come across the need to extend or modify the overly simplistic product and pricing data model that hasn’t been flexible enough to accomodate many real life sales scenarios. Jumping from such a customized environment in production use into Dual-write may not be all that easy, nor will the new data model defined by Microsoft just magically solve everyone’s problems. Dynamics 365 consultants working on data integration between systems aren’t likely going to be out of jobs just yet.
Whether you’ll be jumping into deploying Dual-write or not, everyone needs to educate themselves on the implications of this new piece in the Dynamics 365 puzzle. From the DW documentation, have a look at the chapter “What does dual-write mean for users and architects of CRM products?” to understand why. There will be several new concepts introduced over on the CDS side like “company” and “party” that will affect all apps built on top of it – such as Dynamics 365 Sales, Marketing or Customer Service. Foundational elements like how customers of the organization or the internal units within that organization are defined have different data models in CDS vs. FinOps, so it’s important that everyone working on either side will use the new common documentation for Dual-write as the starting point to try and understand eachother’s perspectives on the world.
There are also other dimensions affecting future solution architecture listed out in the documentation. According to MS, initially Dual-write will be shipped as separate solutions (notice that DW is solution-aware right from GA, highlighting the role of CDS as the cross-product ALM foundation in Power Platform). “In future releases, those solutions will become part of Common Data Service”, which I presume translates to every Dynamics 365 customer getting the core components, regardless of whether they already have Finance and Operations apps deployed in their tenant or not. Data types in CDS are also about to get extended to accomodate the enterprise ERP needs arising from this out-of-the-box FinOps integration.
Of course not everyone is using the ERP solution based on AX, Finance & Operations, Unified Operations, Finance/SCM or whatever name Microsoft comes up next for it. A large share of the customer base will be a better fit for Dynamics 365 Business Central, which isn’t at least publicly yet targeted with Dual-write. Is there going to be an impact to SMB implementations where CDS is part of the solution architecture in some way and is this going to be positive or negative? According to the documentation, “in future releases, concepts in model-driven apps in Dynamics 365 (for example, customer, contact, quotation, and order) will be scaled to mid-market and upper-mid-market customers.” If Microsoft is really aiming to have a Common Data Model that should serve not just all Dynamics 365 products but also software providers from outside the immediate MS ecosystem, there are bound to be trade-offs between the complexity of the required schema for enterprise ERP and the ease of use for how citizen developers can tap into the Common Data Service for building simple business apps.
Finally, there’s the ever interesting licensing aspect, of course. From the Ignite 2019 presentation, we can find the following statement on Dual-write licensing implications:
When Microsoft customers (ISV, Partners, End Users, etc.) possess a valid Dynamics 365 for Finance and Operations license, they are entitled to replicate data between the Common Data Service and Dynamics 365 for Finance and Operations. This entitlement does not include capacity consumed by the replicated data; if necessary, this is purchased separately. This entitlement does not include use rights for the Dynamics 365 for Customer Engagement applications, including Sales, Service, Marketing, etc; if necessary, this is purchased separately.
I’m expecting that the upcoming versions of the Dynamics 365 Licensing Guide would include documentation about the licensing scenarios for Dual-write, as at the time of writing there isn’t any public guidance available yet to address the concerns of multiplexing that inherently arise from the real-time synchronization of ERP data into CDS.
As we know, having Power Apps licenses allows you to read all of CDS data. If you read through the GA announcement of Dual-write and its documentation, you’ll notice that MS is following its own guidance and not mentioning Customer Engagement even once, since CE now refers to on-premises software only. The new term, “Model-driven apps in Dynamics 365”, is a representation of Microsoft app teams themselves ship as the Sales app, Customer Service app and so on. When it comes to the customer specific business apps tailored to their needs, meaning Power Apps, these can be Model-driven or Canvas, ranging from simple “one click” mobile experiences to advanced information worker UI’s for desktop environments. They are all backed by the Common Data Service and running on the Microsoft Power Platform. How will these custom apps be treated in the licensing documentation that traditionally assumes customers would just be using the Dynamics 365 Apps as-is rather than adapting them to their unique needs and real life processes? This remains to be seen.
Now that we are gradually getting over the age old divide between CRM and ERP, I hope the next area of unification would be a common licensing story that doesn’t draw artificial lines between Dynamics 365 and Power Apps. Splitting these into different licensing guides and different pricing publication formats isn’t all that helpful for the customers nor the partner ecosystem. A Common Data Service based on a Common Data Model should preferably be backed by a Common Licensing Model, don’t you think?
