Cloud & DevOps2023

Fragments - Cloud Microservice

A highly scalable cloud-based microservice for storing and converting text and image fragments

DockerAWS EC2S3DynamoDBECSGitLab CI/CDNode.jsExpressJestREST API

About the Project

Fragments is a cloud-based microservice developed for a fictional Canadian automotive company that needed to connect, digitize, and automate their internal systems. The company identified a critical gap: many sub-systems needed to work with small fragments of text and images that were smaller than traditional documents and used various formats.

The system solves real business problems: IoT devices on manufacturing floors produce reports in plain text, CSV, and JSON formats. Factory workers use mobile apps to write status updates that need to be saved as JSON, Markdown, Text, or HTML. Automated cameras on assembly lines capture images of damaged parts for audit purposes.

The microservice provides a highly scalable HTTP REST API that seamlessly integrates with existing systems, uses the company's authorization infrastructure, and deploys to AWS. It supports text formats (plain text, Markdown, HTML, JSON) and image formats (PNG, JPEG, WebP, GIF) with automatic format conversion capabilities.

A key feature is the ability to convert fragments between different formats on retrieval—for example, Markdown to HTML or JPEG to PNG—without increasing storage costs since only the original version is stored. All data is isolated per user with proper authorization.

Key Features

  • HTTP REST API for CRUD operations on text and image fragments
  • Support for multiple text formats: plain text, Markdown, HTML, JSON, CSV
  • Support for image formats: PNG, JPEG, WebP, GIF
  • Automatic format conversion on retrieval (e.g., Markdown → HTML, JPEG → PNG)
  • AWS S3 for scalable blob storage, DynamoDB for metadata
  • User isolation and authorization for all operations
  • Docker containerization with automated CI/CD to AWS ECS
  • Comprehensive test suite with Jest for reliability

Technology Stack

Docker
AWS EC2
S3
DynamoDB
ECS
GitLab CI/CD
Node.js
Express
Jest
REST API

Interested in the code?

Check out the full source code on GitHub.