The 2021 Esri User Conference (UC) gets underway on July 12th and as in 2020 this year’s event is online-only due to the Coronavirus pandemic.
Ahead of the conference each year, Esri publishes its UC Q&A – to save you a little time, here are some highlights that I think might be relevant for colleagues across the Energy and Natural Resources sectors that we work with here at Exprodat and Getech.
Creating a Sustainable Future
The theme of this year’s UC, GIS – Creating a Sustainable Future, focusses the conference on how we GIS users can help create a more sustainable world, through a combination of geographic thinking, holistic problem solving and information integration. Esri’s hope is to inspire and mobilize its users to address the big challenges it sees of climate change and falling biodiversity. For me, this is a welcome shift of focus from the more technology-led themes of previous years – and I’m sure those of us in the Energy sector have encountered these sustainability issues more and more in the past year as the pandemic has accelerated not only awareness of but also activity around the Energy Transition.
As at last year’s first virtual UC, this year’s Plenary has been divided into separate sessions to better support online delivery – both of which are being livestreamed and recorded. Notably this year’s Plenary is divided into 2 sessions, rather than 3 as last year, showing Esri is learning from and optimising its online events delivery.
In the first session Jack Dangermond will update us on Esri’s work and the work of the Esri user community – look out for examples of Exprodat’s work in Renewables in the Natural Resources/Energy sections – followed by staff demos of Esri’s latest technology. The second session will feature 4 keynote presenters who will share stories about their work:
- Paul Salopek, writer and National Geographic Fellow, will provide his own unique perspective and insights on sustainability – he is currently in year 8 of the ‘Out of Eden Walk’, a 24,000 mile (38,400 km) journey that follows the migration of our early ancestors – from Africa to the tip of South America.
- La June Montgomery Tabron, President and CEO of the W.K. Kellogg Foundation, will discuss the steps the foundation is taking to create a more equitable future through mapping and community action.
- Enric Sala, National Geographic Explorer in Residence and founder of ‘Pristine Seas’, will share the importance of understanding where to designate marine protected areas (MPAs), protecting the ecosystem of the ocean and help secure a sustainable future.
- Secretary Wade Crowfoot, California Natural Resources Agency, will discuss the steps the State of California is taking to preserve 30% of its land and water by the year 2030 as part of the ‘30×30’ initiative, and to combat climate change.
How’s Esri doing?
We know the past few years have been tough for many businesses, so how has Esri fared? Esri says its core business has remained strong and stable through 2020 and 2021, and as announced in last year’s Q&A and in line with many organizations, it has continued in its own digital transformation efforts.
Esri notes that “the possibilities for GIS continue to expand”, and not just among GIS professionals, saying that the GIS profession is “becoming recognized across organizations as key digital infrastructure for management, operations, and sustainability”, because “GIS allows people to make connections related to the physical world that might not otherwise be apparent, GIS technology promotes data-driven strategies, strong leadership, and science-based actions. It lets us all see our world for what it is, so that we can take steps to discover what it could be.”
Esri says that growth of GIS is being driven by advances that make it more accessible – e.g. apps, web maps, and location analytics – and Esri sees enormous growth potential in both the deployment of GIS and the opportunities for GIS professionals.
As this use expands, Esri notes that organizations are increasingly developing strategic plans for geospatial infrastructure, placing the world “in the midst of a data explosion. The problem is not so much how to produce data—it’s how to harness and manage it. And with so much of this data having a location component, GIS is providing a logical foundation for organizing, analyzing, and applying all this spatial data.”
As we’ve always known at Exprodat, how to manage your data remains a crucial value contributor to any GIS deployment, and to the quality of decisions that can be derived from it.
Emerging technologies
Esri says it continues to advance GIS and ArcGIS technology by expanding core capabilities, modernizing deployment patterns, and improving business models to support the work of GIS users and enhance the impact they are having on the organizations and communities they serve. Esri quotes the most recent examples of this include the following advancements:
- Geospatial infrastructure – GIS implementations are becoming more interconnected, supporting ever more efficient sharing and collaboration across networks and between separate organizations.
- Pervasive maps and dashboards – Dynamic data-driven web maps, dashboards and GIS apps are becoming pervasive. By way of an example, the Johns Hopkins University COVID-19 dashboard has already been viewed more than two trillion times by nearly half of the world’s population!
- Story Maps – Esri continues to make this technology easier to use and notes that several million (!) ArcGIS Story Maps have now been created with thousands of new ones published every day.
- SaaS implementations – Use of ArcGIS Online and similar web-based systems continue to grow rapidly simply because they make it so much easier to get started with GIS. Esri says it users worldwide have now created over 33 million layers in ArcGIS Online, with as many as 55% of these items shared beyond the hosting organization. Interestingly, these are actually the same figures that Esri gave in 2020, so I would expect the number of layers created to be even higher.
- Real-time GIS – Integration of real-time data into GIS can help organizations be more responsive and effective. ArcGIS can now ingest “almost any kind of streaming data”. Certainly Esri have really beefed up its offering in this space with ArcGIS Velocity (more on this below).
- Hosted imagery – ArcGIS Online is being enhanced to support hosted imagery and analytics, via new SaaS products such as ArcGIS Image (more on this below) which enable you to “host, analyze, and stream imagery, such that it can be easily integrated into all spatial applications”. This should be of particular interest to data vendors everywhere.
- Reality capture integration – New capabilities to capture and process point cloud datasets and transform them into GIS-ready datasets including 3-D meshes, orthophotos and digital terrain surfaces. From some of the work we have done at Exprodat, a lot of the workflows around reality capture seem ripe for efficiency improvements.
- Advanced spatial analysis and data science – Esri says it continues “to innovate on all the key building blocks of spatial analysis and data science: data engineering, visualization and exploration, spatial analysis, machine learning and AI, big data analytics, modelling and scripting, and sharing and collaboration.” The new Data Engineering experience in ArcGIS Pro is one such example, along with continued integration with open source data science tools. Meanwhile, Esri has made enhancements to better support temporal analysis across its products.
- Integrating GeoAI, ML, and deep learning – ArcGIS “offers new levels of automation, pattern recognition, and predictive modelling”. In the past year, Esri has doubled the total number of deep learning models supported within ArcGIS via the ArcGIS API for Python, added new interactive analysis tools to ArcGIS Pro, and released a set of pre-trained GeoAI models that can accelerate AI projects.
- Integrating field operations – Esri’s Field Maps product provides “a completely integrated and single app experience to users who need to capture data in the field, providing them with the power of location and GIS.”
- Expanding developer capabilities – Esri leverages a rich set of geospatial services across its applications and earlier this year, with the release of ArcGIS Platform (more on this below), Esri further opened-up access to these services to enable developers to more easily build focussed applications that access data and ArcGIS capabilities at a massive scale.
Geo-Enabled Systems are a group of Esri products that support focused workflows to more easily accomplish work, and can be used by non-GIS experts. This year, and building on previous announcements on its work with Autodesk, Esri say it is planning to release a new Geo-Enabled System called ArcGIS GeoBIM that gives design and construction project teams the power to easily integrate GIS data with Building Information Modeling (BIM) processes and systems.
GeoBIM is planned for both ArcGIS Online (2021) and ArcGIS Enterprise (2022) deployments and Esri says it will initially offer direct integration to link GIS with BIM data in the Autodesk Construction Cloud.
ArcGIS Pro
Esri notes that ArcGIS Pro “has far exceeded the spatial analysis capabilities of ArcMap”, and anyone who uses ArcGIS Pro will have noticed that Esri have been busy this last year – with 3 releases of ArcGIS Pro since the last UC.
- ArcGIS Pro 2.6 included over 150 enhancements and brought new tools, workflows, and capabilities requested by users, including Voxel layers (a new layer type that creates 3D volumetric visualizations from netCDF data) for helping understand complex, multidimensional data such as atmospheric data, oceanic data, and sub-surface data and models.
- ArcGIS Pro 2.7 delivered new functionality such as layer and feature blending, 3D object feature classes, deep learning exploratory analysis tools, Pantone colors, GNSS (GPS) device location support and mobile geodatabases – a first step in allowing fully interoperable workflows between ArcGIS Pro and ArcGIS Runtime.
- The most recent ArcGIS Pro 2.8 release focused heavily on performance and productivity improvements, and in addition contained a new Data Engineering view (to help you better understand your data and prepare it for GIS analytics workflows) and a Command search (to help you find commands and tools by name or keyword).
One new cool feature that you may not be aware of, is that you can now copy a tool’s Python command directly after populating the tool’s parameters before running it – previously you had to first run the tool in ArcGIS Pro and then copy the command from the geoprocessing history.
ArcMap
Esri’s position on ArcMap is unchanged – it last released a new version of ArcMap as part of ArcGIS 10.8.1 and does not have plans to release ArcMap on 10.9.
The 10.8.x series of ArcMap will therefore be the final series but Esri says it will be supported until March 2026. To help customers transition to ArcGIS Pro (which we strongly recommend!) Esri has created the ArcMap Continued Support page.
Related to ArcMap, 10.8.x was also the last release for the Geometric Network (now essentially being replaced by the Utility Network), with support also running to 2026.
ArcGIS Enterprise
Moving on to ArcGIS Enterprise, version 10.9 includes “exciting updates and enhancements throughout the product in many areas—applications, managing data, and administrative options, ArcGIS Enterprise sites”, along with the following updates:
- Shared editing in distributed collaboration, enabling organizations to collaborate on the same dataset (feature layer).
- A new Home Page editor for responsive, sleek home page design.
- Administrative reports, providing exports of information on organization items and members further analysis and action.
- Tools to migrate services from the ArcMap-based runtime to the ArcGIS Pro-based runtime.
- Support for OpenID Connect and multifactor authentication for built-in accounts.
- New email alerts, such as when your portal machine’s disk space is running low.
- Fine-grained editing settings, e.g., the ability to control editing on a per-field basis.
The new Map Viewer interface (as seen in ArcGIS Online, and powered by JavaScript API 4.x) is now available in ArcGIS Enterprise as a beta version. Esri is targeting that the first full release will be available side-by-side with the original ‘Map Viewer Classic’ by the end of 2021.
As discussed last year, Kubernetes (a new cloud-native deployment option for ArcGIS Enterprise based on the principles of microservices and delivered through containerization) joins Windows and Linux as options for deploying ArcGIS Enterprise. Windows and Linux remain fully viable and supported operating systems for ArcGIS Enterprise, and Esri has no plans for these to be deprecated or removed, saying it will continue to deliver innovations to all three deployment options.
ArcGIS Online
ArcGIS Online has a host of recently added features, including new Home Page customization options; automated scheduling of ArcGIS Notebooks; ‘Collaborations’ between two or more ArcGIS Online organizations; Scene Viewer additions such as elevation profile, shadow highlight, and visualization improvements; and greater control over app access and tabs to help Administrators simplify User experience. It’s great to see the advancements wrt Collaborations – in line with Esri’s own recognition in the power of geospatial infrastructure – and I hope Esri continues to recognise the importance of this work and irons-out some of the wrinkles that currently exist with ArcGIS Online to ArcGIS Enterprise collaborations.
Map Viewer “Classic” and the new Map Viewer (now out of beta for ArcGIS Online) will be available side-by-side “until there is functional parity”. With the new Map Viewer, there are still some key components like the analysis tools that need to be added before Map Viewer Classic will be retired. For planning purposes, Esri says Map Viewer Classic will be available for the rest of 2021, and into 2022.
ArcGIS Image has been in development for some time, but is now very close to release. It allows you to “conveniently host, stream, and analyze your imagery collections through a software-as-a-service solution”. We’ve participated in the beta program for ArcGIS Image and it does look very good – so this one comes highly recommended.
You can now create hosted feature service webhooks in ArcGIS Online. Webhooks are “user-defined HTTP call-backs that are usually triggered by an event, such as editing or syncing a feature or changing the layer metadata”. When that event occurs, the configured source (such as a hosted feature service) makes an HTTP request to the URL configured for the webhook. Essentially webhooks allow you to automate important workflows such as data QA/QC and send notifications to admins or data owners.
Good news for anyone with constraints on where they can host data – Esri have added Asia-Pacific to the list of regions now supported for hosting your ArcGIS Online data, alongside Europe and the US. Great to see Esri thinking globally.
For Developers
ArcGIS Platform is a platform-as-a-service (PaaS) for developers who need to integrate location capabilities into their apps, business systems, and products. It lets developers access Esri’s location services using their APIs of choice—including open-source APIs (such as OpenLayers, Leaflet, and Mapbox GL JS) – and allows developers to create apps and extend ArcGIS products using ArcGIS APIs, SDKs, and app builders. This one is really interesting: I think ArcGIS Platform has the potential to bring Esri capabilities to a huge number of new people, currently working outside of the Esri ecosystem, which could lead to huge expansion in Esri’s customer base.
Calcite is a new offering that provides access to Esri’s design system, a collection of design and development resources for creating “beautiful, easy-to-use, cohesive experiences across apps with minimal effort”. It includes design best practices, icons, color schemes and an accessible web component library with UI elements such as buttons, panels, accordions and alerts. Certainly something we’ll be trying out soon in our own development work.
Moving on to Experience Builder – Esri says that starting with the core and popular widgets, Experience Builder will reach most of the functional parity of Web AppBuilder through incremental releases. However, it has no plans to replace ArcGIS Web AppBuilder with Experience Builder, and it will be running both builders in parallel “into the foreseeable future”.
Capabilities of ArcGIS
Esri continues to evolve its Smart Mapping capabilities, in order to help users more easily make high-quality, data-driven maps. Recent enhancements include Dot Density drawing style, ‘Above and Below’ symbology theme to help highlight trends, automatic symbol and line width adjustment with changing scale (nice!), more color ramp styles, Clustering improvements and (finally!!) the ability to group layers (provided by the new Map Viewer app).
Esri is taking advantage of evolving web technology to deliver these advances. The new Map Viewer, powered by JavaScript API 4.x, has seen a “5-50x improvement” in the number of features you can work with, renderer speeds and network efficiency. This improved performance allows the tools to try, for example, 10 different solutions to quickly find which works best – all in search of better quality. So, Smart Mapping is essentially getting smarter as well as faster.
“Map notes” are now called Sketch Layers and offer a suite of drawing and text tools. These become their own entries in the layers panel, controllable like any other layer in a web map, e.g. they can have pop-ups and a visible display range.
ArcGIS Field Maps was released in Q4 2020 and it is intended to replace the separate Collector, Explorer and Tracker mobile apps for iOS and Android. In addition to providing the functionality of those apps, Field Maps also supports form-based data capture through ArcGIS web map-based smart forms. Esri is also currently planning on integrating the Navigator and Workforce app capabilities into Field Maps. It’s great to see Esri evolving its mobile app offering so quickly, and making the mobile apps more and more useful.
ArcGIS Velocity is Esri’s latest cloud-native real-time / big data analysis capability, released as part of ArcGIS Online last year and significantly enhanced since with a host of new features. ArcGIS Velocity enables users to ingest, visualize, and analyze spatial real-time and big data to gain new insights and make informed decisions. Note that ArcGIS GeoEvent Server remains the on-premise real-time data solution, and is part of ArcGIS Enterprise alongside the spatiotemporal big data store. However, given the complexities of running ‘GeoEvent’ on-premise, I can see a lot of customers taking a look at ArcGIS Velocity as it has the benefit of a much more modern microservices and containerized architecture, having been deployed on Kubernetes.
Finally, some interesting news on new Esri integrations with Microsoft. First up, ArcGIS for Teams enables users to find and share web maps and other ArcGIS Online data from their organization while in a Microsoft Teams online meeting. Secondly, as part of Esri’s work with Microsoft 365, it will be releasing connectors and flows within Power Automate to “help automate regular mapping tasks with your data”. This one looks like it could be quite interesting.
Stay safe!
Sadly, we won’t see one another once again at UC this year, but I hope you all enjoy the virtual event and I hope to see San Diego again in 2022.
Posted by Chris Jepps, COO, Exprodat.