From DataZone to SageMaker Unified Studio: A Cataloging Evolution
Introduction
In previous workflows, many teams relied on Amazon DataZone to share data across AWS. It helped set up domains, publish data, and manage access. Recently, AWS launched SageMaker Unified Studio (SMUS).
This is not just a new name. It is a major evolution. SMUS combines analytics, machine learning, and data cataloging into a single workspace. This shift changes how projects, environments, and permissions are structured.
As I shape my own career path to become a project manager, I view these shifts not just as technical updates, but as fundamental changes in how product and data teams collaborate. In this post, we will explore these structural differences at a high level. In upcoming articles, we will dive deeper into the technical implementation.
The Core Cataloging Value
Amazon DataZone provided a central business data catalog. Producers published assets, and consumers securely subscribed to them. The system handled much of the governance and automated permissions across boundaries.
SMUS takes this foundation and expands it. Instead of cataloging data in one tool and building machine learning models in another, SMUS lets teams do both in the same place. Data engineers and ML practitioners now share the same project context.
Key Architectural Differences
If your organization is planning a move from a DataZone setup to SMUS, there are five major structural changes that product owners and technical decision-makers must understand:
1. Project Structure and Flexible Workspaces
The concept of an environment has changed. In SMUS, an environment is essentially a recipee book for a dedicated capacity. A project can still have multiple environments, but the definition of an environment is no longer restricted to the project itself. It is also no longer locked to a fixed workspace. This gives teams much more flexibility to scale and organize their infrastructure.
2. Permission Management and External Access
DataZone relied heavily on automatic permission management. SMUS introduces a shift in responsibility. Users must now grant themselves permissions to access data assets. Additionally, there is a significant change in the "Bring Your Own" (BYO) strategy. This specifically changes how teams configure and manage access to external services.
3. Unified Cataloging Database
In the past, you might have maintained separate databases for publishing assets and subscribing to assets. SMUS streamlines this approach. The assets used for both publishing and subscription are now merged into a single database. However, if your business logic requires the old separation, you can still implement custom workflows to replicate the DataZone properties.
4. The New Tooling Blueprint
SMUS introduces a new foundational blueprint called the "Tooling" blueprint. This base layer is common to all projects. It standardizes how developer tools and integrations are provisioned across the organization, making it easier for platform teams to manage standard software requirements.
5. Off-the-Shelf Capabilities
Projects now feature multiple ready-to-use capabilities right off the shelf. You have the flexibility to select specific capabilities based on what the product team needs. You also have the option to enable all available capabilities for a single project at once, reducing the time it takes to onboard a new team.
Looking Ahead
These are the high-level structural changes between DataZone and SMUS. The environment abstraction is different, permission models require a new approach, and databases are more consolidated.
In my next series of articles, I will break down each of these five points. I will share the exact technical details and configurations needed to build and manage these new architectures.