A practical system design process

Author: Ansel Robateau 7/2/2024

This document presents a unique method for outlining and developing system designs using specialized diagramming tools, namely yEd Live Diagram Editor and Miro. The primary objective is to provide a practical example and showcase the results of such a process. The intention is to share this information with the public for their review and critical analysis, with the aim of determining its potential value and merit.

I was recently assigned to automate a complex, multi-step process. The project had a diverse group of stakeholders, and the information I received initially was scattered and sometimes contradictory. I was bounced from my supervisor to a product manager, then to another product manager, and finally handed a document detailing the existing manual process.

The complexity of the process, combined with outdated documentation and conflicting communication styles, quickly led to roadblocks. There were disagreements on the direction of the project and the system's architecture. This created a frustrating environment where the project seemed to drag on endlessly.

Realizing I needed a new approach, I took a step back to re-evaluate. I shifted my focus to the end-user experience first. What did we want the users to be able to do? Once I had a clear vision of this, I brought all stakeholders together to establish a unified understanding and agreement.

With the desired user actions in mind, we identified the backend system actions (APIs) necessary to support them. I meticulously mapped out the systems and interactions required for each API. This breakdown allowed me to tackle each system action individually, first determining how to perform them manually.

I consistently sought confirmation from stakeholders, ensuring alignment and buy-in on each step. With a clear roadmap in place, I broke the project into smaller, manageable deliverables, each tied to a specific system action.

This approach revolutionized the project. Implementation became streamlined and efficient because each deliverable had already been validated and agreed upon. It was easy to track progress and see how each piece contributed to the overall user experience.

In the end, I successfully delivered the project. The automated process functioned flawlessly for end users, and stakeholders were thrilled with the results. The experience taught me the value of user-centric design, open communication, and meticulous planning in overcoming even the most challenging projects.

During the design process, I utilized two online tools available to everyone: yEd Live Diagram Editor and Miro.

• yEd Live Diagram Editor: https://www.yworks.com/yed-live/

• Miro: https://miro.com/

Roles of These Tools

When identifying user actions and backend APIs along with their interactions, it's crucial to focus on connections rather than organizing diagrams manually. yEd Live Diagram Editor facilitates rapid ideation with a unique feature: it can automatically organize and format diagrams with a single click using advanced layout algorithms.

Effective communication is vital for a successful project. Miro excels in visually representing information to a team by allowing all users to see the same information in real-time and collaborate on a shared board. Its infinite canvas and ability to embed various media types—such as images, diagrams, screenshots, and shapes—enable the project manager to share every project detail. This includes the overall design, specifications, descriptions, individual steps, examples, and links to reference materials.

Why is This a Differentiator?

The ability to quickly create high-quality diagrams allows users to concentrate on ideas and problem-solving without getting bogged down by manual organization. This capability lets you start from scratch and swiftly represent complex ideas.

Sharing information visually brings the team together, ensuring clear communication and comprehensive project definition. Miro's collaborative features enable documenting feedback and using the overall design as a foundation for creating tasks to implement the project.

Step 1: Gather User Stories

• Collect user stories that describe the desired features and functionalities from the perspective of the end users.

Step 2: Itemize User Actions

• Based on the user stories, identify and list the specific user actions.

• Add these user actions to a diagram, connecting them to the relevant user roles and to each other based on their dependencies.

Step 3: Review User Stories with Stakeholders

• Present the diagrammed user stories to stakeholders.

• Ensure agreement and gather feedback to refine the user stories.

Step 4: Identify Backend APIs

• Determine the backend APIs required to support each user action.

• Connect these APIs to the corresponding user actions in the diagram.

Step 5: Detail Service Interactions

• For each backend API, identify all necessary service interactions (e.g., database queries, calculations, calls to external services, entity interactions).

• Connect these interactions to the backend APIs in the diagram.

Step 6: Create a Full System Diagram

• Develop a comprehensive system diagram that includes all user actions, backend API calls, and service interactions.

• Ensure each element is accompanied by clear descriptions.

Step 7: Validate User Action Implementation

• Manually step through each user action's implementation plan.

• Identify the requirements and technologies needed to make each user action functional.

Step 8: Collaborate and Refine

• Work with stakeholders to review the full system breakdown.

• Incorporate feedback, make necessary adjustments, and secure agreement on the proposed solution.

Step 9: Assign Tickets

• Create and assign tasks (tickets) based on each user action, ensuring clear definitions and responsibilities.

Let's dive into how to use yEd Diagram Editor and Miro in the system design process.

Represent User Stories in a Diagram

1. Create a New Diagram

   • Go to yEd Live and start with a blank diagram. Use the Light Theme for consistency.

2. Add Actors

   • Represent actors with People icons from the palette. Place them on the canvas and label them appropriately (e.g., Sales Associate, User).

3. Add User Actions

   • Select a shape to represent user actions, place it on the canvas, and center the text within the shape.

4. Connect Actors to User Actions

   • Use solid arrows to connect actors to their user actions.

5. Multiple User Actions

   • If an actor has multiple actions, repeat steps 3 and 4.

6. Indicate Dependencies

   • Connect dependent actions with solid arrows (e.g., "log in" to "submit resume").

7. Apply Layout Algorithm

   • Choose an appropriate layout algorithm (e.g., BPMN) to organize the diagram.

Share User Stories

1. Create a Miro Board

   • Go to Miro and create a blank board.

2. Export and Place Diagram

   • Export the yEd diagram as an image and upload it to the Miro board.

3. Share and Collaborate

   • Share the Miro board with stakeholders, discuss the diagram, and make adjustments as needed.

Add Backend APIs

1. Add Backend APIs

   • In yEd Live, add shapes to represent backend APIs. Label them appropriately and connect them to the corresponding user actions with solid arrows.

2. Repeat for All Actions

   • Ensure all user actions have connected backend APIs.

Add System Interactions

1. Add System Actions

   • Add shapes representing system actions (e.g., queries, API calls, calculations). Label and connect them to the backend APIs.

2. Add Systems and Data Sources

   • Represent systems, services, and data sources (e.g., database, API service) with appropriate shapes and add them to the canvas.

3. Add Data Objects

   • Use the Entity with Attributes shape to represent data objects being communicated.

4. Create Connections

   • Use solid arrows for cause and effect connections and dotted arrows for data retrieval.

5. Repeat for All APIs

   • Ensure all backend APIs have connected system interactions.

Presenting the Plan in Miro

1. Layout Diagram

   • In yEd Live, apply a layout algorithm (e.g., BPMN).

2. Save and Export

   • Save the diagram as a .graphml file and export it as an image (using the sketchy style for a more approachable look).

3. Import to Miro

   • Upload the exported diagram to Miro.

4. Add Project Details

   • Draw a frame around the diagram in Miro, adding a project title and description.

5. Highlight User Actions

   • Select each user action in yEd Live, view related shapes in the neighborhood view, and expand the view to include all related connections. Use screen capture to copy these views to Miro.

6. Repeat for All Actions

   • Ensure each user action and backend API is clearly represented.

7. Describe Diagrams

   • In Miro, frame each user action diagram and use text blocks to describe them.

Documenting Implementation Validation

1. Validate Workflows

   • As you validate each backend API workflow, add proof and references in Miro frames.

2. Illustrate Documentation

   • Use screenshots, code snippets, documentation links, note cards, etc., to document the implementation.

Collaboration and Refinement

1. Discuss Plans

   • Use the Miro board to discuss implementation plans with stakeholders. Evaluate details, make adjustments, and reach an agreement.

Ticket Creation and Assignment

1. Map to Tickets

   • Map each backend API workflow to tickets based on complexity. The discussed, validated, and documented plan should facilitate straightforward implementation, clearly attributable to user actions.

By following these steps, you can effectively use yEd Diagram Editor and Miro to support the system design process, ensuring clarity, collaboration, and thorough documentation.

To illustrate this design process, let’s design a simple resume submission service.  The scope of the service is limited to the following criteria.

1. The user must be logged in

2. The user must be able to submit a resume in the form of a Microsoft Word document, a pdf or an image (tiff, png)

3. We are not concerned about the user interface, just the core functionality.

4. We are not concerned about how the resume is processed by downstream systems.

5. The user authentication and session management service already exists.

Gathering the user stories

The user stories are:

User Stories

As a Job Seeker, I want to login so that I can access the resume service

As a Job Seeker, I want to submit a resume so that I can apply for a job

In this example, we have one actor, Job Seeker and we have two user actions, login and submit a resume.

Using the tools

Let’s start with a blank yEd canvas.

By using a consistent visual language, we can quickly review and understand complex information. Diagramming the full system workflow ensures that all connections between functionalities are discovered and accounted for, providing a comprehensive view of the project.

Sharing this information visually in Miro allows all stakeholders to grasp the scope of work and the resources involved. The Miro board becomes a vital reference point for those implementing the plan and serves as a communication hub for all stakeholders, facilitating ongoing collaboration and alignment.

The system design process is a critical component of successful project development. By leveraging tools like yEd Diagram Editor and Miro, we can streamline this process, ensuring clarity and collaboration at every stage. Diagramming user stories and their corresponding backend APIs helps us visualize complex workflows and identify dependencies early on. Sharing these diagrams in Miro enables stakeholders to understand the project scope, provide valuable feedback, and reach a consensus efficiently.

The consistent use of visual language across these tools not only aids in quick comprehension but also ensures that every detail is accounted for, reducing the risk of overlooked requirements. The Miro board, enriched with diagrams, documentation, and real-time collaborative features, becomes an indispensable asset, guiding the project from planning to implementation.

By following the structured method outlined in this article, teams can achieve a thorough and unified approach to system design, ultimately leading to more successful project outcomes.