The experimental feature for connecting Power Apps Canvas apps with Git version control is pretty amazing. You can read this blog post from my FF colleague Timo Pertilä to see how it looks like for app makers: Power Apps and Git version control.
What doesn’t feel quite as amazing for a low-code guy like me is the steps needed for configuring the local Visual Studio Code client to connect with a GitHub repo. Timo has also blogged about these steps.
I’m not sure how many people are yet aware of the really magical feature that GitHub offers: the github.dev web-based editor. In short, what this allows you to do is skip all of the local configuration steps and launch an instance of VS Code directly from a GitHub repo – in your browser. Here it is in action:
(Grab this MP4 video if the GIF animation doesn’t show up properly.)
What are the steps needed? First, we completely everything in Timo’s first blog post. Create a GitHub repo, then enable the experimental feature in Canvas app settings. Note: you will now have irreversibly linked your app to GitHub, so don’t do this for any “real” app just yet.
After linking the repo Power Apps will automatically copy all of the app source code there. Open up GitHub.com and browse to the /Src folder to find YAML source code files for your Canvas app screens.
Open a YAML file you want to edit. Then, it’s time for MAGIC! Press the period key (.) on your keyboard and watch VS Code load up in the current browser window, with the file you were looking at over on github.com.
You’ve now got immediate access to the wide variety of editing features of VS Code, to perform updates on any file in that repo you’re in. How does github.dev do this? Here’s the answer:
The web-based editor runs entirely in your browser’s sandbox. The editor doesn’t clone the repository, but instead uses the GitHub Repositories extension to carry out most of the functionality that you will use. Your work is saved in the browser’s local storage until you commit it. You should commit your changes regularly to ensure that they’re always accessible.
A simple example of the benefits is that we can visually track all the changes performed on our YAML file. Then, as we’re ready to commit it, just add a commit message and everything will be synced to the repo.
Whenv we’re back in the Power Apps studio, clicking on the sync button will retrieve the latest updates from GitHub:
And there we have it! A new value for the “app name” label that we edited in VS Code, with nothing but GitHub.com and a single “.” keypress.
Could we one day do all of this directly from within the Power Apps Maker portal / studio? Well, as you can see, Microsoft has a lot of the infrastructure already in place to make the online browsing and editing of Canvas app source code technically possible.
Throughout my whole professional career, I’ve worked with technology that caters to information workers. Sometimes I’ve been the power user of these tools myself, then a manager responsible for defining the details how these systems should work. For the past decade I’ve been the guy who actually puts the technology pieces together to deliver that system.
I have never written code in traditional programming languages for a customer project. Sure, I’ve done plenty of work that isn’t accomplished via point & click GUIs, but none of it resembles what the Wikipedia definition of computer programming says. In short: I don’t code.
Every year it feels like I’m sinking deeper and deeper into the technology domain of Microsoft business applications, and at the same time it’s less & less likely that I’d start writing code for a living. The rise of low-code development platforms (LCDP / LCAP) like the Microsoft Power Platform means that there are more & more people like me out there. Citizen developers are taking over many application development tasks but they don’t do what traditional programmers used to do.
The impact from this paradigm shift is potentially massive. Computer programming isn’t going away nor decreasing as a professional activity. However, application development as a task is being separated from the domain of programming. The output of what “code” used to create is therefore being democratized:
Democratization of technology refers to the process by which access to technology rapidly continues to become more accessible to more people.
In this article I want to write down a few observations and thoughts on this topic, which I’ll hopefully get to drill deeper into in future writings.
From code-first to low-code
Things have moved along surprisingly fast. Just three years ago there was a small debate in the Dynamics 365 community on whether all functional consultants should know how to write code. Neil Benson’s excellent article called “Three forbidden words: I don’t code” popped up in my Timehop stream a while ago to remind me about this. (I must stress that despite of the title, Neil actually argues against the need for every consultant in the project team to write code.)
Today in 2021, this whole question on whether every IT worker should invest time and effort in learning a “real” programming language felt to me like it must have been from a decade ago. To put things into perspective, we have to keep in mind that this original debate took place in 2018 , at a time when the fusion of Microsoft Dynamics 365 and PowerApps (the “no space” era) had just been announced. MS had only just begun the process of throwing BizApps consultants and citizen developers into the same pool of technology. It was a very different world still.
Fast forward to 2021. If you look at what the Power Platform community has been debating about recently, I’ve seen a lot of concern from people with software development background & skills. “Isn’t Microsoft valuing coders anymore?” is a question that can rightfully be asked when looking at the common theme of marketing messaging coming from Redmond. It’s a stark difference to just 3 years ago for sure.
Already earlier, many customers who have previously been burned by problematic CRM implementations full of custom code have begun expressing their preferences in subsequent projects. “Avoid code based extensions and leverage OoB configuration options wherever possible” has not been an uncommon requirement to hear from them.
Now when Microsoft is advertising how powerful the low-code options can be, it’s going to be even harder to suggest code-first solutions to customers. At least you need a clear justification for why custom code is necessary. This can have unfortunate side effects, too, where the avoidance of coding leads to an even harder to maintain maze of configuration spaghetti. Different cooks, same dish.
I’m not so sure that it’s wise for MS to go out and shout to the world at this point that “Power Fx is the world’s most used programming language”. Also, I wouldn’t personally say that my Excel skills have gotten me that far in creating formulas for Power Apps Canvas apps. It takes a lot of tutorials and browsing through the formula reference to get into grips with Power Fx.
Yet there are unquestionable benefits from using the “Think Excel” mantra as the backbone for developing this new programming language for the low-code era. Not just for sharing the same function names, but rather for ensuring the focus of Power Fx remains squarely on the power users that can make Excel sing. Not those wannabe programmers who eventually want to move from the safe sandbox of Power Fx into something without the training wheels. Of course one still might need to have the programmer’s mental model for producing complex apps with Power Fx regardless:
The biggest problem with Excel is that it presents itself as a programmer’s tool, with a programmer’s value system. Programmers can make Excel sing, while normal humans can just muddle through. The solution does not lie in programmer mental models. 24
We need to keep in mind that Power Fx is not aimed at the audience who writes code for a living. Microsoft is not expecting that developers with skills in other programming languages would switch over to Power Fx. Rather the purpose of (eventually) open sourcing the details of the language implementation would appear to be in enabling other parties outside Microsoft to adopt Power Fx for their products.
Despite the ubiquitous role of Excel in the business world, we should not assume this spreadsheet software to be the only place where future app developer generations will gain their experience from. Gaming platforms like Minecraft or Roblox with their big focus on user generated content could be considered equally important sources from where the inspiration to become a digital maker is first put into the mind of an individual.
After all, being a guru in Excel formulas isn’t perhaps the strongest indicator of being a potential business app maker. The no-code generation is less likely to dive deep on a single tool, rather they are fearless in combining new services together on the fly to reach the outcome that they need. Separating this ability to grasp new technical tools from the ability to write algorithms in traditional software code is perhaps where the true democratization will take place.
Professionals vs. Citizens
The terms we use for grouping people into different teams in the current articles that cover the phenomena of low-code and business applications are somewhat problematic. On one side we have the “classic” developers under the banner that says Professional Developers, while the other crowd represents the “modern” way of creating apps and has been given the Citizen Developers banner.
“And now, ladies & gentlemen, LET’S GET READY TO RUMBLE!!!!”
That is exactly how we should NOT depict the situation. There are no winners and losers in this game. We haven’t invented a silver bullet that solves all business problems. There’s no eternal glory to be found from being “all code” or “no code” in everything you build. Yet it’s obvious that we’re not talking about a homogeneous group of developers here. Some of the platforms and tools used may be shared across, but the roles remain distinct.
The amount of low-code business applications that organizations of all sizes in every industry will soon have in their hands means that we’ll see a growing number of employees working 100% on these. If low-code becomes your full-time job, you’re not just a citizen or a hobbyist. You’re a professional working with advanced technologies and complex processes. The only thing that separates you from the “classic” definition of a Professional Developer is that you don’t write much code (aside from things like Power Fx).
Microsoft seems to have already reduced their use of the Citizen Developer term. This doesn’t necessarily mean that it would not continue to exist in the discussion, as the idea of anyone being empowered to create apps still is a powerful way to get the message across. It’s more likely that the terms used to describe the different roles within the “fusion teams” that deliver Power Platform based solutions will need to be adjusted.
When a business professional starts handling more and more technical tasks in the app delivery process, I don’t think this makes him/her a Pro Dev nor a Citizen Dev. It is the area in which I personally operate and yet I can put a label on it. Luckily it doesn’t make my work any less valuable. I’ve always lived as the secret agent between business & IT, so I can easily remain to be that mysterious Agent X. As the army of these agents grows, though, they’ll become a lot less secret with their presence within organizations.
Are they simply “Makers”? Time will tell.
Where democratization may lead us
I think what’s happening today with software resembles something that has previously happened with media. Two decades ago we saw the Web 2.0 phenomenon that moved the online world forward from a static web towards a social web. The elements and the actors aren’t all that different from what the move from code-first to low-code apps is dealing with.
Content creation exploded when the ordinary people (citizens) were given tools and channels to shift from just being content consumers to the new role of content creators and publishers. That was a huge transfer of power. For me personally, my decision to start this blog back in 2008 and to become an active participant in the Twitter community around Microsoft business solutions have been far more powerful decisions than anything I’ve ever had the chance to make at my actual day job.
If the traditional media was run by Professional Journalists then what emerged from the social web & social media revolution might as well be called Citizen Journalists. No longer was there any formal training nor editorial processes required for getting your content out there. It had a profound impact on our society as a whole, in both good and bad. The forming of new citizen driven communities around topics like business apps is a very mild example on that scale, but it’s where I’ve been able to observe the phenomenon in the closest possible detail.
What happened with human communication in the Social Web revolution could very well happen with human-to-computer communication in the next uprising. On a high level, I can’t actually see that many differences between what WordPress did to words and what Power Platform aims to do to apps. Different context yet the same underlying dynamics of how the traditional top-down model is replaced with a bottom-up approach.
One specific insight that could be drawn from this analogy between the social web and low-code movement is the ratio between different user roles in each. Even if anyone could technically start a blog these days, relatively few people have done it (and even fewer have managed to stick to it over the years). Most are merely content consumers, and some are active in commenting or sharing content.
It’s very likely that we’ll see a similar distribution when it comes to low-code apps in the business world. Only a small share of those with access to the tools will ever become consistent app makers – and that’s perfectly fine. It’ll still be big enough figures to disrupt the traditional usage of code.
Opportunities and threats of low-code
Just like the rise of Facebook or Twitter didn’t solve every problem related to media, we aren’t going to reach the promised land of digital transformation merely through low-code application platforms. Many of the arguments heard from Professional Developers on why the actions of Citizen Developers will create a flood of new problems are in fact well founded concerns.
Change management and version control don’t suddenly become a non-issue. Application lifecycle management (ALM) isn’t automatically taken care of. Architecture design and documentation can’t be skipped. Just by moving certain parts of the software development lifecycle from code based tools to graphical configuration tools and formulas like Power Fx we’re not magically making all the other parts irrelevant.
Some of the problems encountered with traditional business application delivery will only be amplified by the low-code model. As the volume of apps grows, as we have more parties involved in their development process, as the backgrounds of these individuals will be more varied – we’ll run into scalability challenges for sure.
This just highlights the need for new innovation. Reaching into the existing ProDev toolbox isn’t likely to be the answer – just like you couldn’t adopt the practices from a traditional media newsroom and apply those into social media platforms.
Be it the creation of content or apps, any model of manual governance and control that relies on A) the creators to follow specific rules and policies, and B) human editors/gatekeepers to enforce them, isn’t going to scale very far. Algorithms must be put into use. AI needs to be given the opportunity to help us in everything that goes into developing and maintaining great business apps.
Can’t put the genie back in the bottle
It’s time to accept the fact that this thing won’t go away. What low-code represents is not any specific service or tool that today allows you to do A, B and C – but not yet D, let alone E. It’s all about the journey – how to move closer and closer to Z, one step at a time.
Success is not defined by whether you would ever reach Z without writing a line of code. What low-code platforms aim to do instead is to keep grabbing the low hanging fruits of app development, one by one. Focusing solely on the “what’s missing compared to code-first” aspect is therefore not very beneficial in understanding the actual value that has been derived from different generations of low-code solutions, be it RAD tools or application platforms like XRM.
Sure, there aren’t many new apps being built on top of MS Access or Lotus Notes anymore (hopefully). Does that prove it was a bad idea to build any of them in the first place? Of course it doesn’t. These tools may have only been good at A or B back in their days, yet there was significant business value to be gained for getting the crude apps out there to the users who needed to manage digital information via them. Besides, even if you built a “proper” app via custom code in the golden era of Access or Notes, you would have likely needed to re-build that solution a few times anyway to stay relevant with modern client technology and new business requirements.
The evolution of software keeps pushing us towards higher levels of abstraction. It would be strange to think that the direction could ever turn and we’d see a large scale return to artisan software being carefully crafted by coding every line by hand, reducing the use of libraries and APIs that offer shortcuts to those soulless programmers that just want to meet customer requirements quick & easy.
Early days still
I’ve had a great opportunity to dive deeper into the concept of low-code during the past year, ever since we founded a company that focuses 100% on Microsoft Power Platform based solutions. One key insights from all my investigation work has been that we’re not yet very close in finding common ground on what low-code actually means. Neither the vendors, tech media, consultants nor customers seem to have a clear understanding on where to position low-code in the greater IT scheme.
Academic research on topics like end-user development (EUD) / end-user programming (EUP) seems to have mostly stopped almost 10 years ago already – before the invention of the term “low-code”. One study from December 2019 specifically calls out the lack of recent research activity “There are very few publications related to low-code aspects and they are from the last two years, demonstrating its emerging trend.”
Since it currently appears to be mostly the vendors like Microsoft, OutSystems or Mendix who are pushing forward their agenda, with help from analysts like Forrester in building up the hype, it’s hard to know what percentage of the low-code and citizen developer stories out there are based on facts. Sure, projections like these make it sound like it’s already an amazing success story:
Gartner forecasts worldwide low-code development technologies market to grow 23% in 2021.
Forrester analysts estimate 75% of all enterprise software will be built with low-code technology this coming year 2021.
It must be great for anyone investing in low-code to be able to present such growth percentages, to prove they’re betting on a winning horse. Still, I feel like we’re only starting to write the greater story behind all this exciting technology. Democratizing the tools through which our digital world is built one app or one automation at a time – that’s gonna be a bigger deal than just the amount of VC money being pumped into no-code/low-code startups.
At the end, it’s not even about which tools and platforms will win. Looking at both the investments as well as progress made by Microsoft into/with Power Platform, I have very little doubt that they will be taking low-code into mainstream for real. What’s the truly interesting part is thinking about all the ways in which this may change the business user behaviour, the role of IT departments, the market dynamics for professional services – basically everything around the low-code platforms.
If that’s something you’re also interested in talking about in more detail, perhaps you’d want to reach out to us at Forward Forever. Or if you feel like there’s something I’m missing in the above thoughts around the democratization of code, I’d sure appreaciate any comments in the box below.
Application Lifecycle Management, ALM, is a topic rarely covered in the mainstream pitch for how low-code/no-code solutions are transforming app development practices. This is understandable, since the initial time-to-value, from an identified business need to a working application (or at least a functioning prototype of one) is where the major difference to the traditional software development initiatives are easiest to demonstrate. It’s the “five minutes to WOW” effect that gets most of the attention.
Relying on configuration and formulas over custom code does promise also a lower maintenance overhead for the post go-live phase where the apps are used and evolved gradually to remain in line with the business requirements. There is less to worry about the potential impact of changes in each of the underlying components that keep the app running when you’re subscribing to a service like Power Platform that aims to abstract away many of the moving pieces in Azure and other related cloud services.
You shouldn’t assume life to be completely worry free, though. If you don’t plan ahead on what type of work is needed after the creation of your low-code app, then a nasty surprise may lie ahead for someone, somewhere, at the most inconvenient time. In the worst case your organization has become reliant on an app that no one owns & no one knows how it works, only to realize that your processes have already stopped due to errors caused by a change either inside or outside the system. Updates to the cloud components or changes in APIs used via Connectors, for example. A slower yet equally frustrating problem can arise from identifying a future change you’d really need to implement for the app in question, yet you don’t have the means to make that happen in practice.
The rise of low-code application development and the forecasted arrival of 500 million new apps in the next 5 years is making IT organizations very aware of the need for setting up governance models to address this new reality. Having baseline knowledge of what apps are created, by whom, for which audience, for what purpose, through what technologies, on what data sources etc. is often the discussion that needs to be done first to analyze the current situation. A great tool for facilitating this is the Power Platform Center of Excellence (CoE) Starter Kit, which gives you an overview of the big picture for Power Apps and Flows in your tenant. To go deeper and actually manage the lifecycle of an individual (potentially business critical) application, we need to start discussing ALM.
What is this ALM thing anyway?
Microsoft has published fresh new documentation on ALM with Power Platform that could be viewed as a similar Starter Kit for this topic as the aforementioned CoE. It briefly covers the high level summary of what ALM means:
ALM is the lifecycle management of applications, which includes governance, development, and maintenance. Moreover, it includes these disciplines: requirements management, software architecture, development, testing, maintenance, change management, continuous integration, project management, deployment, and release management.
A very broad concept then. What I like most about this intro chapter is the definition of the goal of ALM: “driving efficiency through predictable and repeatable software delivery”. That’s the essence right there. Anything that reduces the differences between how individual app makers / developers perform common tasks within the whole lifecycle of a specific business application will improve the state of your ALM practices. Those with a software development background may immediately venture into thinking about various DevOps methods and tools at this point, whereas I’m always most concerned about how the business can get a grip of what exactly is happening to their applications over time. Who built what, based on which specification, when was it deployed & where to, how were the changes documented, what impact did it have on others systems, how were users educated on the new features, and so on.
Microsoft’s ALM guidance is mostly revolving around the technical level specific to Power Platform, meaning how solutions and environments should be used. If you haven’t heard it before, then the solution framework that was born in 2010 for the XRM era has been actively expanded to cover more & more areas that are not a part of the Dynamics 365 Customer Engagement on-premises server. Canvas apps, Flows, AI Builder models, Power Virtual Agent chatbos, Export to Data Lake configs, pretty much everything in the Power Platform is now manifested as solution component. More is on the way, based on how Matt Barbour has stated things like Power BI and integration services being “on the radar” for the Power Platform solution system.
The expanding technical capabilities and scope of components shipped via solutions is of course great news. However, as we are now talking about the platform for every developer, not just pro devs, my number one question about Power Platform ALM is not “will it scale up to new products and features” but rather “will it scale down to new types of apps that don’t follow the lifecycle of a CRM system?” So, there’s two dimensions I feel will be critical for the ALM story to succeed: 1) unifying the mechanisms to manage configuration & code across the different app teams within MS Business Applications Group, and 2) democratizing healthy ALM practices by making them accessible to citizen developers. The first one is well underway with the solutionization of everything, but how about the second one?
Managing your “source low-code”
The fundamental challenge with why ALM tools aren’t already a part of every Power Apps maker’s or Dynamics 365 customizer’s core workflow is that they don’t exist within the context where these people do their work. This audience doesn’t need Visual Studio for anything, nor would there be anything familiar with the terminology used by version control systems like Git. This is alien technology from Planet Developer, which the no-code app makers would have to learn specifically for the purpose of managing how their configuration deliverables are moved from one environment to another. Compare that to the “export solution as .zip” / “import solution .zip” actions availble in the Power Apps Maker GUI that gets the job done for them. Many of these app makers surely understand the value that a formal deployment process and solution versioning could offer, but the barrier has been much too high for them to make that leap.
There have been tools for pro-code developers working with XRM solutions for many years already, some provided by MS and many built by the community. Today there are official Microsoft Power Platform Build Tools available for Azure DevOps, which offer readymade tasks like solution export, import, unpack, pack, publish, with more features to come. Rather than tweaking PowerShell scripts, it would seem like there’s a way to visually configure the cloud to do some ALM magic for our low-code apps. Sure, this is not within Power Apps directly, but could this be considered just another admin portal that the citizen developers might learn to navigate?
Setting up an Azure DevOps pipeline that pulls solutions from your dev environment, commits them into a repository for version control and then deploys that same solution to test or prod still isn’t exactly a next-next-next process. However, following a Power Platform specific tutorial, even a non-developer like me can get this up & running in a few hours. MVP Nick Doelman has written exactly what I needed, so if you’re interested in building your very first ALM pipeline, I recommend you to try this out:
Combining this with a recent update to the Power Apps Build Tools that also introduced support for MFA protected environments (meaning you can log in with an Application User identity), I was able to get our company’s “CRM” solution’s customization changes from from Dev to GitHub and Dev to Prod. Here’s what it looks like now when adding a new field on the accout entity:
“Woo-hoo, we now have proper ALM in place!” Well, not quite. All that we really achieved with this was the possiblity of doing it “right”, meaning there are DevOps pipelines that could be used for handling configuration changes automatically instead of manually by individual low-code developers. Until I actually go and remove all Power Platform service admin roles from my colleagues, anyone of them could still go and mess up the production customizations directly. So, proper ALM is only partially about the tooling and automation – you still need to the exact same thing as without DevOps, meaning defining the processes and agreeing on the practices for how the business applications are managed.
Regardless of this, it does makes perfect sense for low-code app development projects to evaluate if they would get benefits from leveraging automation tools like Azure DevOps. You’ll need to have the solution configuration stored somewhere anyway, so why not put it in a version control system that’s designed for “real code” and gain access to things like viewing the changes of individual solution components – rather than just dumping the files on OneDrive/SharePoint. Who knows, taking that step might even unlock further opportunities to automate and harmonize your application management processes.
We should keep in mind that automation is a double-edged sword. In theory it can save you a lot of time, yet in practice it may often turn into an additional time sink of its own. The legendary webcomic XKCD beautifully illustrated this dilemma of Automation, which I think is especially true here in the ALM discussion. The tasks that you’re trying to automate need to happen on a frequent enough basis for a long enough time before you could achieve a positive ROI from investing in setting up your pipelines. A single citizen developer building a one-off app may not yet meet that threshold, so preferably in your automation scenario there should be a team of developers who will do several planned releases for the app in question.
ALM for a team of low-code developers
Microsoft likes to promote the stories of Power Apps Champions who have managed to make digital transformation a reality for their organizations by building low-code apps that would have earlier required a team of software developers. This marketing approach higlights the role of an individual hero, but from an ALM perspective the dependency to any individual person’s heroic actions and secret knowledge is rather a problem that should be solved. Once that initial app idea and first PoC starts turning into something business critical, you should have a team in place that can ensure its continuity.
This team may not resemble your typical software development team. Yes, it’s quite likely that there will also be people with programming skills either working in the team or closely with it, but the primary competence on the technology side for this team is in assembling the features and components of the chose application platform into solutions that solve a wide variety of business problems. They are less likely to be all working with the single code base of a large application, doing pull requests, commits and all that Git jazz, simply because a fair share of Power Apps aren’t going to be built that way at all.
If you look at Power Apps Canvas apps today, they are essentially a document made of JSON that cannot be simultaneously worked on by multiple developers. This is very different from a traditional app project on the Model-driven side (let’s say in a CRM context) where one person could be building the customer data management features while another team member is working on products and order management. All with their own dedicated entities, having so little overlap that simultaneous work within the same dev environment is often doable. Even though Components promise to offer a more modular way to build capabilities for the Canvas world (and Model, too), I suspect that the client side experience of a single app will typically remain a “one maker show” in the near future.
Sharing elements and patterns across different apps built and managed by the team is, on the other hand, in a much greater role in the low-code approach. Here components, template apps and other reusable pieces of configuration can be of high value. Power Apps began its journey as a citizen-first app creation experience yet is gradually accumulating more of these features that mean every app doesn’t have to be a unique project. We’re getting more & more possibilities to see beyond the Maker GUI, like “peek code” in a Flow or use Monitor to debug Canvas app formulas. However, based on what Microsoft representatives have publicly said, it doesn’t sound likely that these low-code tools would evolve into allowing app makers to directly manipulate the configuration files in VS Code, for example. There are community tools like CanvasAppPackager from MVP Daryl LaBar that will help app developers understand complex Canvas apps, but these are NOT meant for building or modifying apps.
Even in Model-driven apps the list of supported modifications for customizations.xml is fairly limited. So, keep this in mind when thinking if the pro-dev tools and ALM practices that are based on direct source code manupulation are going to be a fit for the low-code app developer team or not. Your Power Platform experts need to be primarily fluent with the admin tools from MS and the community (always start from XrmToolBox), plus they’ll need excellent skills for using collaborative tools like Teams within your org to share their lessons learned. Once you’ve got this side is covered, then you can move onto Azure DevOps and the likes.
Environment management is another big topic for low-code apps on Power Platform. Having a separate CDS environment for every developer working on their corner of an application is Microsoft’s recommended approach to keep changes isolated. As your number of “ALM enabled” apps in the tenant will grow, the total number of environments will grow even faster. Assuming a theoretical 5 GB size for the CDS database of an environment, the $200/month storage cost itself wouldn’t actually be that significant if it saves a few hours of work from those developers. What you don’t want to happen, though, is having dormant dev environments consuming CDS capacity for many months with no actual deliverables from them – especially when your CDS quota is defined on a tenant level with no control available for reserving it to specific apps/environments.
There are remedies to this problem. The “governance way” is managing the environment creation/deletion though a process that provides transparency to environment usage and enables allocating costs accordingly, for example via custom apps built on top of the CoE Starter Kit. The “ALM way” is provisioning on-demand environments from the source code in your repo via DevOps automation, then scrapping them after the active development work stops. While I sort of fancy this idea of “everything as code” where all of the infrastructure and configuration is managed via the same pipeline, I can’t help thinking that the most difficult thing about environments for business app development isn’t necessarily the configuration or the code. It’s the valid test data needed for making the app come alive. Yes, of course you could also script the perfect test data set to be generated alongside the environments, but again, we’re taking the level of effort needed in achieving deployment automation onto a very non-citizen developer level and approaching the XKCD scenario.
Managed Solutions: what do you REALLY want to manage?
Whenever Microsoft talks about ALM, usually you’re going to hear their statement on “use only managed solutions in production” sooner later than later. It’s not surprising that also the latest guidance, such as the MBAS 2o20 sesssion on Advanced Application Lifecycle Management capabilities lists this high up on the ALM Health Check list:
In the user community around XRM there’s been an endless debate around whether unmanaged or managed solutions are really the way to go, and we’re not going to find the ultimate answer to it here in my blog. Personally, I’ve always had a hard time understanding the exact benefit that a customer organization would get from self-imposing the limitations of managed solutions onto themselves. It just sounds to me like installing a different lock with a dedicated key onto the window blinds in every room of your house. Expensive, overly complex, and very likely to leave you in the dark when someone forgets which key was used for which lock. It can also give you a false sense of security, as the main door to your house is still left completely unlocked, because Microsoft doesn’t yet offer a way to protect your production environments from the deployment of unmanaged customizations by anyone with sysadmin rights.
In the context of low-code specifically, if you don’t have a seriously mature ALM process in place, I’d say you’re fairly likely to hurt yourself when playing with the sharp objects that are managed solutions. This is why during my 10 years of working with Dynamics CRM, XRM and now Power Apps, I haven’t yet deployed a single managed solution inside a customer specific project. On the other hand, I have worked with removing managed solutions created by the customers’ previous partners, after the ALM process has broken down for one reason or another (“not existing” being one of them), then replacing them with an unmanaged layer of customizations that has been a better fit for their overall CRM landscape. Yes, those other partners probably defaulted to following MS instructions on using managed solutions in every production environment – this guidance remained the same since CRM 2011 days, after all.
Please do note that I’m not saying managed solutions wouldn’t serve a purpose. When you are building an actual app product that is meant to be deployed across multiple environments for different customers, managed solutions are of course the way to go. They offer the container for keeping your stuff separate from the stuff delivered by other app vendors, when deployed into the single CRM system that’s used widely across the enterprise by different user groups for different processes. Any proper commercial product development initiative should also be able to absorb the ALM overhead from environments, automation etc. and turn that into increased revenue from being able to ship software releases faster and more reliably as time goes by, thanks to the upfront investment made. It all makes sense here, but this is a very different landscape than how I perceive the typical use cases for low-code apps within organizations to evolve as Power Platform becomes more widely adopted.
Remember: you’re no longer building “one place for all customer data”, rather you’re often creating several targeted apps for very specific purposes and audiences. Most of these app experiences will likely be fully developed by either savvy citizen developers, an in-house team of Power Apps champions, or an external consultant brought in to get that job done. The app lifecycle can be very short, even targeted to be used in one-off events (or under temporary disruptions such as COVID-19 social distancing), whereby some apps may actually be “disposable by design”. These are pretty much the opposite of monolithic CRM systems where a wealth of different ISV solutions need to peacefully co-exist throughout the years. So, whichever solution strategy you choose for your Power Platform environments, make sure it solves a real problem that your low-code app developers and admins are facing, rather than directly adapting practices that have been designed for the ALM needs of pro-developer teams working full-time on shipping a sizeable amount of custom code alongside the low-code application platform configuration information.
A different approach to ALM: Power BI deployment pipelines
The broad umbrella of Microsoft Power Platform covers also technology that isn’t (currently) built on the solution framework: Power BI. Since it is in practical terms more part of MS Data Platform, the mechanisms used on the BI side are understandably quite different than those originating from the citizen-dev PowerApps (back when it was still spelled together) or CRM systems. What makes this an interesting area to observe is that from what I’ve understood, the traditional development of reporting solutions hasn’t exactly followed the typical software development path either. Checking in reports to source control systems may not be the natural workflow of a Power BI content author or data analyst, so the barrier for adopting advanced ALM practices is likely to be pretty high for this audience, too.
Power BI deployment pipelines is a feature that has recently been launched in preview. Labelled as an ALM solution, as also Power BI now packages report content into “apps”, this functionality is actually baked right in to the native user interface of the PBI service:
Reading the introductory text gives us an idea of how this feature is positioned:
Deployment pipelines help enterprise BI teams build an efficient and reusable process by maintaining development, test, and production environments. BI creators can incrementally transition new or updated content between environments, reconfiguring them with the appropriate data connections and permissions. Pipelines are very easy to use and take only few minutes to setup. No previous expertise or coding skills required.
Wouldn’t it be nice if we could take the term “BI” from that paragraph, replace it with “App” and have a similar story to tell for low-code business app developers? A targeted experience that would be readily available for each & every customer environment right out of the box would certainly be the most effective way to democratize healthy ALM practices.
You’ll need Power BI Premium capacity for using deployment pipelines, so the reporting ALM story is aimed towards the enterprise audience. If there ever was a native deployment pipeline functionality for Power Apps, I’m sure the usage would require consuming some type of paid for capacity from the Power Platform product subscriptions. I believe this would still be a lot easier approach for selling the idea of why customers need to invest in having proper ALM infrastructure and capabilities in place, since it wouldn’t be just an extra layer of billable hours added on top of app development project budgets but rather something much more tangible.
For the first time ever at Microsoft Build conference, the Power Platform was presented right at the start of Satya’s keynote this year! Woo-hoo!
Of course this time last year there wasn’t yet the name “Power Platform” to even reference at Build. We had only just seen the merger of XRM and PowerApps into something that was a bit of a puzzle to communicate to partners, let alone customers. Well, the puzzle hasn’t exactly been solved yet, but it is still quite remarkable how far we’ve come in one year already.
Last year’s sessions at Build 2018 were mostly about introducing the concepts like Common Data Service to a .NET developer audience that probably had zero hands-on experience with any Dynamics product for the most part. Not a whole lot of noise was made about this entry into the #MSBuild space. Fast forward to 2019 and now the vision of uniting pro developers with “every developer” is already touted at the keynote sessions. Not just that, but Satya is saying that recent re-architecting of Dynamics 365 on top of Azure infrastructure and services should be examined as an excellent reference for anyone who’s planning to build their own products on SQL Azure.
During the week of Build, the product team behind Citizen Application Platform (“CAP”) puts aside their Citizen caps and pulls on the pro dev hoodies to talk about topics like solution management, PCF component development, Azure Functions, DevOps pipelines and all the nerdy stuff that would scare away the folks who normally create PowerApps. It’s inevitable that as the tools for app makers get more mature the next barrier to world domination will be in getting not just the IT admins to build the necessary automation and governance around Power Platform in enterprise environments but also in finding a way to make pro-coders play with low-coders.
If you look back at XRM, then there’s really nothing new about this division of roles. It has always been the case that code illiterate business analysts do the point & click configuration work for data models and business processes, while the XRM developers spend their time with the SDK enabled client-side extensions, server-side logic and system integration tasks. Fundamentally what the Power Platform does is it enables everyone to level up in their game. Application design on the UI level and interfaces to connected data sources can now be handled by those business analysts who are willing to learn new low-code tricks. Similarly, the developers get to break free from the boundaries of the IIS and SQL Server boxes, to harness the amazing power of The World’s Computer (Microsoft’s nickname for Azure) to hook into new AI services and crunch the contents of The Real Common Data Service.
If the app builders are about to step up their game, so must the sales machine of Microsoft. The big push from Redmond is now on ensuring that an ecosystem will emerge on top of Power Platform. The new partner program for Business Application ISVs, lead by Steve Guggenheimer, is trying to make a bigger splash by combining the earlier models of Azure Marketplace and the Dynamics 365 focused AppSource into a single channel that could actually serve the grand vision of a no-cliffs development platform. As always, you should check out what The Other Steve has to say about the upsides to the new program, before making your conclusions on whether it’s just a new tax on ISVs or an opportunity worth pursuing for a growing number of MS partners.
To summarize the announcements and buzz around Power Platform at Microsoft Build 2019 conference, I’ve compiled this handy lil’ Twitter Moment for you to enjoy:
Two weeks ago Neil Benson wrote an excellent article on LinkedIn, as a response to a claim that everyone working as a functional consultant in a Dynamics 365 project team should also know how to write code. This really resonated with me and I shared the article, along with a bit of commentary of my own. My post, in turn, started to gain quite a lot of traction on the LinkedIn feed. It looks like we had touched upon a topic of great interest within the network of CRM professionals.
If you read my post above, it pretty much summarizes the main points I wish to bring out in this discussion:
There is much more to ensuring Dynamics 365 project success than being able to do hands-on software development.
A big chunk of the value delivered by Microsoft’s cloud platform is not relying on custom code development.
The remaining chunk that is custom code driven work should be left to professional developers and not copy-paste heroes.
There have been some very good arguments written in the comments section of the LinkedIn post, so I urge you to also view them for gaining greater perspective on the topic. I see a lot of resemblance with the “should designers code” meme in the original assumption that functional consultants would be able to do their jobs better if they also were familiar with developing custom code. Much of what has been said in the heat of that debate between coders and non-coders probably applies here, too:
Alan Cooper’s four part article covers the dilemmas in general software development teams with such great insights and depth that I won’t even attempt to dive in the same direction here. In practical terms of a CRM project, when it comes to the availability of skills and experience needed in carrying out the wide range of tasks during the project’s lifecycle, the manager in charge of resources is always going to need to do some juggling. The less you ask the team members to juggle between completely different kinds tasks, the more likely they’ll able to focus on actual customer value generating activities rather than keeping all the balls in the air simultaneously. Here’s how Neil Benson put it:
“Asking a functional consultant, with no formal computer science education or experience as a professional coder, to find and copy someone else’s JScript from Stackoverflow and paste it into your Dynamics solution is asking for trouble.”
I don’t touch code myself but I do try to get my hands dirty on a wide variety of technologies. For example, today I spent a few hours getting familiar with Azure Service Bus, testing how without writing a single line of code I was able to push plugin execution context data into a cloud based message queue and work with it via a GUI in Azure Logic Apps. Now, I would never recommend myself as a person you should hire to set up your production ESB, but I do feel like I need to have a deeper understanding of these technologies than I could gain just by watching through the flashy demos in Microsoft keynotes. Seeing the different sides and reading/hearing what those with deep expertise on a particular technology have to say about it, that is essential. Going and actually trying to step into the shoes of an expert for that technology – probably an unnecessary detour.
Skimming the bits from the top without diving deep into the dirty details of writing real code might sound like being afraid of all the complexity that awaits beneath the surface. However, the lack of code in a solution doesn’t mean that the complexity of the solution couldn’t be high. The CRM Tip Of The Day post “The story of the small change” perfectly illustrates the way in which individual configuration items built entirely via the graphical tools provided by MS can have dependencies that require you to have a rigorous testing process in place. It’s not just the changes of a system either, since you can easily use workflows and business rules to build a level of complexity into your business logic that doesn’t reveal itself in the test scenarios the consultant behind the design might have thought of. Whether you’re delivering the end product via code or configuration, bugs will sneak in there and eat away a part of your life – be it before or after production deployment.
The argument for doing things via point’n’click configuration “because it’s so much faster to build” shouldn’t be the one and only argument. Yes, it is often much, much faster to put together the first iteration of a solution via graphical tools. This in itself can be a huge value for the business because you can validate the proposed solution quickly with the relevant stakeholders. Often you’ll only get to the actual requirements when demonstrating live parts of what the initial requirements said. Now, the main reason why Real Developers are often afraid of what “citizen developers” might come up with when given quick tools for building no-code apps is that they may lack the experience of seeing the full lifecycle of a business application. They’ll mistake the first PoC as being equivalent to the final production solution, with little thought given to the work that still remains ahead. This experience isn’t something you must necessarily gain via writing code yourself – a no-code functional consultant just needs to gain sufficient exposure to the various stages of the process via working as a part of the CRM project delivery team.
I think we all need to be able to see “beyond code” when building solutions on top of platforms like Dynamics 365 Customer Engagement. Going full-on no-code in your design approach is probably going to unnecessarily limit the value that could be gained from an extensible application platform, forcing you to unsustainable workarounds and resulting in poor UX for the system end users. At the other end of the spectrum, always resorting to your custom built components before having a thorough analysis of the configuration capabilities and complementary cloud services found within your ecosystem of choice will likely increase the solution’s TCO and put the long term reliability of your complex business application at risk as changes in related processes, personnel and technologies occur over time.
Please feel free to leave comments on how you see the role of code & no-code work evolving in Dynamics 365 projects.
In my previous blog post I presented the various different meanings that an App can have in Dynamics 365 Customer Engagement. Now that we’re aware of this jungle, let’s grab a machete and start making our way deeper into the heart of it, to understand how a system customizer can survive in there.
Before there was Microsoft AppSource for Dynamics 365, the methods available for distributing apps in a generic sense were pretty basic: you downloaded a zip file (or several) from a location provided by some party, then navigated to the solutions menu in your XRM environment and started importing them. When there were updates to those apps, you needed to repeat the procedure. If there were some other configuration steps needed in getting the application properly set up, you had to read the friendly manual and complete those. In a more tech savvy environment the Package Deployer might have been used here, but that was hardly a task for the accidental CRM administrator.
What AppSource aims to change in the Dynamics 365 app distribution process is similar to what the smartphone app stores did a decade ago, i.e. simplify the steps for the customer and also provide a better channel for app developers to deliver their updates. When you go to AppSource and choose to either install a free App or start a trial on a paid one, the next screen will provide an instance selector to determine where in your Office 365 tenant you want to put this App in. Also presented are the checkboxes for agreeing on both Microsoft’s as well as the ISV’s legal terms.
From here you’ll be taken into Dynamics 365 Administration Center. This part of the process nor the UI of the admin center isn’t very intuitive, so let’s pause here for a moment. While you’ll land on the Solutions view of an Instance after clicking on “Agree”, on the logical level we should be paying attention to the Applications tab instead. The chosen ISV (or MS) App will have been added as a row in the applications list, which applies to the whole tenant. In this example we see that North52 Business Process Activities is now available in our tenant. It doesn’t have any configuration options in this UI, but the Microsoft apps like Portal Add-on or Voice of the Customer both have an additional “Configure” button that is accessed via this Manage Applications screen.
If we click back to the Instances tab in the admin center, select one of our instances and click the Solutions icon on the right side, we’re now presented with the list of solutions available to this instance via the AppSource delivery channel. It is not the same as going to your XRM instance and clicking Settings – Solutions, as there can be more solutions within that instance. For example, the organization specific solutions that you’ve created as a container for your own customizations. Not even the managed/unmanaged status of those solution has anything to do with what’s shown in the admin center, because whatever zip files you imported directly into your XRM instance as a solution is only visible from within the XRM UI.
The solutions list in the admin center is also different in the way that it shows also the solutions you haven’t installed in the instance. These are applications that someone, either MS or your D365 admin, has made available in your tenant and possibly installed them into some other instance (a test sandbox, for example). To get them installed you don’t have to go to AppSource, rather you can start the process from here.
What makes this view so relevant for the Dynamics 365 instance administrator is that here’s where you’ll see what solutions have upgrades available. In the above example, Microsoft has released a new version of the Relationship Insights solution. Since they don’t want to accidentally break your dev/test/production orgs by changing the solutions on their own, they are rather giving you the controls to click on the “Update” icon for the particular instance when you’re ready for it. This same process is applied also for third party ISV solutions to deliver updated versions of their apps.
Now when we have deployed the app from AppSource and the Solutions view in the Dynamics 365 Administration Center for our chosen instance shows the status as “installed”, let’s use the Office 365 app launched to navigate to our Dynamics 365 start page, meaning home.dynamics.com. And… there’s nothing new here. Even if we click the “Sync” button to refresh the My Apps view, our AppSource app doesn’t appear. What gives?
At this point we need to take a step back from the UI and think about how these different components relate with one another. On the highest level we have AppSource, which is more of a marketing UI for products. From there we get Applications into our Dynamics 365 Administration Center. These manifest themselves as single solution rows for an instance when viewed via the admin center, but they can actually contain N separate solution files (look at Dynamics 365 Portals, for example). Finally, these solutions may or may not contain Apps – from 0 to N. This diagram illustrates these four conceptual levels and their relationships:
In our example we’ve installed North52, which is an administrator/customizer tool designed for “building simple or complex business rules using point-click editor, eliminating C# and JavaScript coding”. In short, it’s an app for configuring apps, but it’s not a business app in itself. That doesn’t mean it wouldn’t need a UI, of course, but the Command Bar shortcuts and the dedicated home page web resource with navigation options quite frankly is much better suited for this type of a power user tool than the new Unified Interface apps that are supposed to work even on 4″ mobile phone screens.
This brings us back to the App Module concept that was briefly mentioned in my earlier blog post. Before V9 and the Unified Interface there wasn’t so much benefit in building separate Apps for different functional areas of the XRM platform, as we had the one master UI for the instance available anyway. When the features are migrated over to the new Unified Interface, basically everything must be an App. In v9.0 we’ve yet to see how the more complex admin features will be implemented as Unified Interface versions, so currently it’s a somewhat jarring experience of 2011 meets 2018 for the system customizers.
Even when all the actual business application functionality has moved over to Unified Interface, there will still be many scenarios in which presenting an AppSource app as a Dynamics 365 App Module App doesn’t necessarily make any sense. UI extensions like Checklists will not have much use outside the actual business entity in which they are used. Any app that connects to an external web service to enrich the contents of Dynamics 365 records mainly needs a configuration admin UI somewhere. Sure, there’s nothing stopping developers from using the App Designer to define an App for their solutions, since all you technically need is a single HTML web resource to publish an App with a single menu item. However, separating the tool from the XRM instance in which it lives isn’t going to make the UX of configuring features any easier, so I’m not really hoping for the app clutter to increase this way.
Both the AppSource marketplace and the App Module in Dynamics 365 Customer Engagement provide significant improvement on how the business application features can be presented to business users and decision makers. What they don’t do is completely remove the need for Dynamics 365 system administrators to understand how the various layers and parts of the application platform are wired. XRM will likely remain an environment that’s just inherently more complex than an iPhone screen with its pretty app icons lined up just the way the single device user likes to see them.
The first new component of the upcoming Dynamics 365 platform that has reached a stage of public preview is the Microsoft Common Data Model (CDM). Available via PowerApps, CDM can be provisioned in your Office 365 tenant with only a few clicks, so there’s little reason for not having a look an early look at it. In fact, you only need to sit back and relax while watching CRM MVP Scott Durow walk you through a first look at the Common Data Model:
So, there you have it! That’s what CDM looks like when accessed via the PowerApps web management UI. Any questions?
Yeah, I actually do have a couple.
How will this work with CRM and AX?
What we have available in the preview is pretty much the most straightforward part of the very big puzzle to put together, meaning a database on Azure with some preconfigured tables and data model management tools. We do not yet know much about how the Dynamics CRM and Dynamics AX functionality will be linked to CDM as part of the Dynamics 365 cloud platform, so there’s plenty room for speculation, which honestly is mostly what I’m about to do here. In a way I’m just continuing on the theme of my previous post about Dynamics 365 and its potential implications for XRM, to pass the time as we wait for Microsoft’s plans to be revealed in more detail.
Right now the only way to push data into CDM is a Flow. If you’ve ever played with automation tools like IFTTT or Zapier, then you’ll quickly grasp the idea of Microsoft Flow. The application itself shouldn’t be underestimated just because of its current simplistic demo scenarios that usually are along the lines of “when a new row is added to a SharePoint list, send an email to this address”. Built on top of Azure Logic Apps, there’s actually a next generation BizTalk type of cloud integration platform under the hood, which should provide plenty of future potential for advanced messaging solutions to orchestrate business processes across a number of different systems.
Once Dynamics 365 Enterprise arrives and gives us the features of CRM and AX in one seamless cloud environment, there’s naturally going to be a need for something a lot more than a “build your own” type of Flow integration. Keeping the Sales and Operations apps of D365 in sync with the customer and transaction data managed in the process of making an delivering a sale involves a fair amount of business logic. If you’ve ever designed and developed a custom integration for this type of a scenario, you’ll know the requirements can quickly grow a bit hairy. Assuming Microsoft can come along and say “we’ll take care of that hairy part, don’t you worry about it” then who could resist it?
The reason CDM exists is that there will be more than one physical database in the Dynamics 365 suite. It’s not all XRM, which means you can’t find the Operations app entities inside your CRM solution files. For the business processes to work seamlessly, someone needs to keep those database closely in sync with one another. From reading through the Common Data Model tutorials, we can see that at least as of now, Flow is not the system that can handle it:
“Today, when you use Microsoft Flow to import data or export data, it is not a full synchronization service. Whenever an object is added to one service, it will be imported into the other system. However, that means if an object is deleted from one system it will not be deleted in the other system.”
So, the sync part is still in the “To Be Implemented” bucket. So is security, since the passing of a record from CRM to CDM via Flow will not carry over any details about who should have the rights to do some CRUD work on it. Again, it may not sound like such a mission impossible to build. However, if you’ve ever faced the requirement in a Dynamics CRM project to implement SharePoint document library integration with account records that includes not just linking the folders but also enforcing the account access rights on the documents, you’ll know the struggle is real. Sure, a collaboration solution like SharePoint has very different security concepts than a system designed for structured business records management like CRM or ERP. But if Microsoft hasn’t been able to offer OoB synchronization of access rights across Dynamics CRM and SharePoint despite of the clear business demand for it, maybe we’d be foolish to expect that it will all be seamless inside the Dynamics 365 world either.
The thing here is that unless the solution provided by Microsoft is going to be fairly advanced, it might not be an actual solution. It’s like the old saying from the dawn of the internet:
Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.
When confronted with the need to integrate processes across two different cloud business applications, there’s always the danger of someone rushing into thinking “I know, I’ll build a database in the middle to unify the process data”. So we end up with three cloud business applications… Now, I’m not saying that Microsoft wouldn’t have the type of application architecture masterminds working on the Dynamics 365 platform that can solve these complex problems when developing a new product. I’m just afraid that things may still turn out a bit more complex in reality than the marketing pitch for the new product launch might lead people to believe.
What limitations will this impose on customization?
The one reason why many of us love the capabilities of the Dynamics XRM platform is the awesome flexibility it offers us to customize the application to meet the specific needs of customers. And by “customize” I actually mean “configure”, since these days you can build such amazing features for business users without writing a single line of Javascript or C#. With Dynamics 365 now promising to deliver so many preconfigured apps for different departments’ needs, as well as making them all work together, I bet some of us are thinking about whether there’s a potential threat to the platform’s flexibility buried in the new approach Microsoft is taking. (more…)
To gain tighter control over what you distribute in solutions and solution patches, use solution segmentation. With Microsoft Dynamics CRM solution segmentation, you can export solutions with selected entity assets, such as entity fields, forms, and views, rather than entire entities with all the assets. To create the segmented solutions and patches, you can use the CRM user interface, without writing code.
Sounds pretty cool, doesn’t it? I’m sure there’s hardly a CRM customizer out there who wouldn’t have encountered the challenges with the lack of granularity when it comes to how Dynamics CRM defines some of the solution components to be inseparable from their parent entity. For example, if you just wanted to add a custom chart or modify a system view you always had to include the whole entity into your solution to distribute it from your development environment to other CRM orgs. With this type of “all or nothing” approach, it’s very easy to end up overwriting customizations in the target organization for components that you had no intention of modifying. The worst part really is that you can never be sure if you’ve done some damage after clicking on “publish solution”.
The TechNet article immediately starts talking about concepts like patching and cloning solutions, so it’s important to note here that these really are mostly relevant only to people who ship managed solutions. If you’re an ISV developing an add-on product or work in a large enterprise developer team with automated release processes, this will surely be what you do all day everyday. If, on the other hand, your work is more focused on customizing and configuring Dynamics CRM environments in projects that don’t contain a large amount of custom code and only target a single customer specific production instance, I bet you’re dealing a lot with unmanaged solutions. I know I am, since the price you pay for deploying unmanaged solutions is most often a lot lower than what you’d face when committing to the path of the managed solutions (and crossing that scary point of no return while at it).
What makes the new solution segmentation concept important to understand for any CRM customizer is that each and every one of you will come across it when adding default entities into your solution in CRM 2016 and beyond. After you select an entity like the opportunity, for instance, you’ll be presented with a brand new screen asking you to “select entity assets to include in the solution”:
This is where you now can pick just the components that you wish to ship with your solution once you export it out from the development environment. You could choose just a specific set of fields, for example, or go to the views tab and click the top left corner of the grid to mark all the assets of type “view” to be included in the solution. Want to see how it can be used in practice? Okay, let’s go ahead and pick from the opportunity entity just the Sales Pipeline chart into our solution, so we can modify it.
Let’s say we want to change the pipeline chart to use weighted revenue instead of estimated revenue values. So, where do we get those fields? You create them! CRM doesn’t ship with a weighted revenue field OoB but instead it contains the tools for building it, thanks to the great calculated fields feature introduced in CRM 2015. We’ll therefore need to create a new currency field and set its formula as “estimated revenue * (probability / 100)”. Even though we’re referencing other fields in the formula, there’s no requirement to include them in our solution, as these system fields will be in place in the target environment regardless of our actions. What we might want to pay attention to is something that the solution segmentation engine doesn’t worry about, which is including the system generated “Weighted Revenue (Base)” field into the solution, as that’s where the base currency version of our calculation will be automatically stored. Better safe than sorry! Furthermore, the new solution segmentation model means we can remove any individual component from the solution if we later deem it unnecessary (as shown below).
Now that we’ve got our Weighted Revenue field, we can update the Sales Pipeline chart to use this new value. To test the segmented solutions functionality in more depth, let’s also pick another system component into our package: the Sales Activity Social Dashboard. We’ll update the elements of this dashboard and see how they transfer into the target organization. Once we’re ready, let’s export the solution as an unmanaged package like we’ve always done. We’ll get the warnings on missing dependencies (again, like we always do), but since the whole exercise is about updating just selected pieces of the CRM puzzle, we don’t need to worry about any of those.
Upon importing the solution .zip file into the target org, there are no differences to how things worked with non-segmented solutions. If we want to peek inside solution details before hitting publish, we’re only presented with the high level components, which means there’s no way to tell at this point if the package contains the entire opportunity entity or just a couple of fields from it. If I was importing an ISV solution received from an external source, I would actually like to know what’s included on a very detailed level, but for now we’ll just need to keep on trusting the publisher.
Let’s see what happens to our updated dashboard now. As a starting point I have a CRM Online demo environment from my previous post regarding “preferred vendor solutions” offered by Microsoft. As you can see, one of these solutions (probably FieldOne) has turned the flat CRM 2016 funnel chart into a retro 2011 style 3D version. Yeah, that’s precisely why solution segmentation makes a big difference, as you no longer need to overwrite things just to add new things.
Once we import our new solution and publish the changes, we’ll see that the funnel chart has now been restored to the flat version that we exported from another CRM 2016 org. More importantly, the stages are now different as we’re mapping the opportunities into the pipeline based on our custom field Weighted Revenue instead of Estimated Revenue. The Top Opportunities chart has also been replaced by another custom chart I included into the solution. Looks like our segmentation is working great so far!
Picking the entity assets one by one is perfectly fine for a scenario where you are deploying a very specific change like the one presented here. However, when you aren’t just performing minor tweaks to an existing CRM environment but rather deploying a brand new CRM system for a customer, you’re likely going to be touching a very large number of components. For instance, this week I’ve been working on a deployment project where the current sprint focuses on sales process management functionality. The number of tweaks I’ve done during two straight days of CRM customization is pretty massive, when you start from an out-of-the-box Dynamics CRM environment and turn it into something that actually works the way the users expect. If I would have had to plan in advance which components I need to be touching, or alternatively adding them into the solution one by one as I realize they need to be customized, I would have probably spent ~20 days instead of 2 while waiting for the CRM Online customization dialog windows to open.
What I’m saying is that while having the full granularity of solution components available to you is awesome, there are common scenarios where you actually may want to keep on working the way you’re used to – adding the whole big entity chunk into your solution. Let’s say I’m building a “base customizations” solution package in a development environment which should contain all the core data model and UI customizations for a new CRM org. I know I’ll be messing with the opportunity entity big time, so I want to ensure every change to the OoB configuration is covered. Instead of picking the individual assets, I can tick the box in the top right corner to “add all assets” into the solution at once.
Nice, now I’ve got the best of both worlds! My base customizations package looks like things were before CRM 2016 so I’m comfortable with building the customer specific solution with it. So, with that I’ll conclude my feature overview of a CRM 2016 highlight…
“Wait, hold on just a moment there! Are you telling me there’s not a single “gotcha” discussed in this blog post? This is Surviving CRM! You never do a happy path walkthrough of CRM features without finding some issues in them.” Okay, you got me. I wasn’t really going to stop there. Let’s keep exploring this segmentation path a little deeper, shall we? (more…)
A few people have asked me for recommendations on what books they should get if they want to learn about the ins & outs of the current Microsoft Dynamics CRM 2013 version. Even though there’s a wealth of blog articles out there that study specific features and an ever growing library of content produced by Microsoft themselves, there’s not been a whole lot of material in traditional book format that would have covered the latest CRM 2013 functionality. Well, now there is a great title available that I can recommend: the CRM 2013 QuickStart.
How do I know the book is any good? Because I wrote a part of it! Aside from shameless self promotion, I can honestly say that the writing team behind this book is quite an extraordinary league of CRM experts:
If that’s not a group of CRM MVP’s you’d trust for advice on how to work with the platform then I don’t know who you would!
What exactly does the book cover then? As the title suggests, it’s not a complete A-Z of each individual feature included in the Dynamics CRM 2013 platform. Neither is it meant to be “my first Dynamics CRM manual” for people who are unfamiliar with any version of the product. Let me borrow some of the official intro text for the book here to explain the reason for its existence:
The CRM 2013 Quick Start is a first look at Microsoft Dynamics CRM 2013 and all the new features that have been included.
In the CRM 2013 Quick Start you will find details that can help administrators, customizers (functional consultants) and developers; not to mention power business users wanting to know all the details the admin never tells them. If you run CRM in the cloud or sitting in a server room at your office the information is useful.
This book is targeted to someone who has some CRM prior experience. By that we simply don’t spend any time explaining the basics of Microsoft Dynamics CRM from a beginner’s point of view. That said, the information in this book would still be useful on your journey to become proficient.
Imagine that you’re someone who’s started their Dynamics CRM journey with an earlier version like 4.0 or 2011 and you’re now faced with the upgrade project for moving the solutions onto the latest CRM 2013 version. Is this the right book for you to gain an understanding of what’s new & what has changed in the platform? Absolutely! How about if you’re a system administrator or a customizer that has some exposure to the new version (via CRM Online perhaps) but are looking to ramp up your knowledge about the platform for future projects, enhancements or admin tasks. Will the CRM 2013 QuickStart help you get up to speed faster than searching for random articles online? You bet!
Now, this is actually the first book that I have ever written content for and it makes me immensely proud to have managed to make my debut in such a prestigious crowd of co-authors. My personal contribution to this title focused on describing the founding principles of how to design a great user experience for the CRM solution that you wish to deliver to your end users. While some of the topics I covered in the book are specific to the latest CRM 2013 version, many of the solution design guidelines are actually universally applicable to any Dynamics CRM version, representing best practices that I’ve personally learned over the past decade of working with the product. I’m really glad to have been given the opportunity to present them in a format that allows for a different type of discussion than your typical blog post.
Being a newbie in authoring content for books, it was also a valuable learning experience for me. Although I’ve been writing down my thoughts on all things Dynamics CRM on this blog for six years now, the project of producing close to 50 pages of content on a given topic to create a coherent book chapter that can stand on its own was still a very different kind of assignment. Let’s just say that I have new found respect for authors that have managed to create entire books for new software products on their own.
The great thing about my writing project was that it provided me a really concrete reason to dig into the details of the various new customization options that Dynamics CRM 2013 offers and experiment with different scenarios that I’m likely to encounter in real life customer projects. As they say, the best way to learn a new topic is to teach it to someone else. If this content that I’ve put together as a part of my own learning process then ends up helping also other members of the Dynamics CRM community to discover better ways to solve customers’ problems with the application, then I consider that a win-win result.
If you mostly work with configuring the Microsoft Dynamics CRM solutions via the customization UI with its graphical point & click tools instead of writing custom code in Visual Studio, you may not keep such a close eye on the changes introduced in the CRM SDK. After all, the term SDK stands for “software development kit” and you may not consider yourself as the target audience if you don’t actually develop any new software that would interface with Dynamics CRM.
The What’s New page in the CRM SDK documentation is a place that would be very useful for also non-developers to have a look at whenever updates to the product are released. Apart from the end user guidance material published on CRM Customer Center, the SDK is pretty much the official place where changes to the platform functionality are introduced. With the latest CRM 2013 Service Pack 1 release (a.k.a Leo) now available, there’s a long list of new and improved features to review. One new addition that didn’t receive a mention on that page, though, is the Configuration Migration Tool, which is a very handy tool if you need to transfer metadata entities across different CRM organizations.
What Are “Metadata Entities”?
In theory the CRM solution files should contain all of the configuration data about your CRM organization. This is what is often referred to as metadata, which could be described as “data about data” in short. As an example, the list of possible values for the Relationship Type attribute on your account record would be considered metadata, whereas the actual account records referring to one of these values would be “real” data stored in the business entities. To illustrate the difference, a blank CRM organization containing only your customizations would not contain any records for the business entities (accounts, contacts, opportunities), but it would contain all of the records in the metadata entities that would be necessary for executing the customized business processes, since otherwise the configuration of that environment would be incomplete.
In practice there are often valid reasons for why it makes more sense to store this type of metadata in a custom entity of its own, instead of using option sets. This provides a whole different level of flexibility in terms of what type of metadata and how much information you can store. Maintaining the list of values can be done without the need to update the customizations. Filtering available records to the end user can be done via standard security roles and BU’s. Building dependent lists on the form can be achieved via the filtered lookups feature. You can leverage the new Quick View Forms to show data from related parental records. And so on.
One problem with using these type of metadata entities is that the data cannot be packaged into a CRM solution file. While you can easily export any changes to the standard customization components and move them to a different CRM organization, getting the metadata entity records from one org to another requires a bit more effort. As a result, keeping the different development, test and production environments in sync requires specific attention to be paid on these metadata during the release process.
Introducing the Configuration Migration Tool
The CRM 2013 Service Pack 1 release includes the new Unified Service Desk (USD) application that relies heavily on storing its configuration data into these metadata entities. In order to support the systematic deployment of USD solutions configured for the customer’s environment, Microsoft has introduced new tools in the version 6.1 (a.k.a. CRM 2013 SP1) SDK. These are the Configuration Migration Tool and the Package Deployer tool. While the latter one is surely useful in deploying larger ISV solutions, I believe it’s the Configuration Migration Tool that can benefit a far wider audience. Also, since you don’t need to run Visual Studio to work with the tool, I’m personally more comfortable discussing its features in my blog as a non-developer who generally steers clear from anything that isn’t close to point & click.
In short, what the Configuration Migration Tool (from here on referred to as “CMT”) allows you to do is 1) choose the entities that you want to migrate data from, 2) export all the records from those entities as a .zip file and 3) import it into a different CRM organization. There’s a great illustration available in the SDK article describing CMT, presented below:
“Awesome, now we have a built-in tool for CRM data migrations!” Well, actually, no. While CMT can move a set of records from several entities in one package and maintain their relationships, it has not been designed for the purpose of business data migration. For example, you cannot filter out any of the records to be transferred from the source organization, nor are there any guarantees for the tool’s performance when moving very large sets of data. Here’s what the Dynamics CRM product team has to say about the tool:
Let’s see how the tool works in practice. To test the features, I created two custom entities for storing my configuration data: “Category” and “Subcategory”. A category can have many subcategories (1:N relationship), but in order to make the experiment more interesting, I also added a lookup on the category form that allows selecting the default subcategory to be used. So, we have both a 1:N and N:1 relationship between our entities, as shown on the Category form below. For the purpose of testing I added category A with subcategories 1*, 2, 3 and 4, then category B with subcategories 1 and 2*, with the asterisk determining the chosen default subcategory (notice the duplicate names in the subcategory data, we’ll get to that later on).
After launching CMT, choosing to define the schema (step 1) and entering the connection information to log into my own development CRM 2013 server via IFD (hint: enter the server URL with the org name as “org.domain.com” and the user name as “user@domain.com, no other connection option worked for me), we’re presented with a screen for choosing the components to be included in the schema. We can pick from entities included in a specific solution. After selecting the entity, we can either choose which fields we want to transfer data from or just be lazy and add the whole entity into our schema definition. Save and Export will give us an XML file with the contents from the included entities, fields and relationships.
The next step is the data export. Here we’ll choose the schema XML we just created and a target file for the data export. Hitting the Export button will start the process, report the progress for each entity in the CMT app window and finally produce a .zip file containing the data for all the components defined in the schema file.
The last step is to connect to a different organization that already has the same schema. So, if you’ve not yet transferred your solution files into the target organization, now’s the time to do that, since CMT will not modify the organization schema, but rather just assume that it is identical to the source organization as far as the selected entities and fields go. (more…)
