Software Architecture Document: Learnille Server

Software Architecture Document: Learnille Server

1. Introduction

1.1. Purpose
1.2. Scope
1.3. Definitions, Acronyms, and Abbreviations
1.4. References
1.5. Overview of the Remainder of the Document

2. Architectural Representation

2.1. Architectural Goals
    - Modularity: Achieved through NestJS modules for distinct features.
    - Scalability: Leveraging technologies like Node.js, PostgreSQL, Redis, and BullMQ.
    - Maintainability: TypeScript for type safety, consistent structure via NestJS.
    - Security: JWT-based authentication, role-based access control (CASL observed), Helmet for security headers.
    - Testability: Structure supports unit, integration, and e2e testing (Jest setup observed).
2.2. Constraints
    - Technology Stack: Primarily NestJS (Node.js/TypeScript), PostgreSQL, Redis.
    - Reliance on specific third-party services (e.g., Paystack, Novu, Sentry, New Relic, Cloudflare R2 / S3, InstantDB/Triplit, ClickHouse, Shlink).
2.3. Key Architectural Decisions & Rationale
    - Choice of NestJS: Provides a robust, modular framework for building scalable server-side applications with TypeScript.
    - Modular Design: Enhances separation of concerns, maintainability, and team collaboration.
    - Use of TypeORM: Offers a powerful ORM for database interaction with PostgreSQL.
    - JWT for Authentication: Standard and stateless approach for API security.
    - Asynchronous Processing with BullMQ: For handling background tasks and improving responsiveness.
    - Centralized Configuration: Using `@nestjs/config` with environment-specific files and Joi validation.
    - Event-Driven Elements: Use of `@nestjs/event-emitter` for decoupling components.

3. System Overview and Context

3.1. System Context Diagram
    (Conceptual - to be described textually for now)
    The Learnille Server acts as the central backend API for various clients (web application, mobile applications). It interacts with:
    - Databases: Self-Hosted PostgreSQL (primary), Redis (caching/session/rate-limiting), OpenSearch (search), InstantDB/Triplit.
    - Payment Gateways: Paystack, Flutterwave.
    - Notification Service: Novu.
    - Error Tracking & Monitoring: Sentry, New Relic.
    - File Storage: Cloudflare R2 (S3 API compatible) / MinIO.
    - Authentication: Google & Apple (for social login).
    - URL Shortening: Shlink.
3.2. Major Components
    - API Layer: Exposes RESTful endpoints (NestJS Controllers).
    - Application Core: Business logic encapsulated in NestJS Services and Modules.
    - Data Access Layer: TypeORM repositories interacting with databases.
    - Infrastructure Services: Wrappers or integrations for Redis, BullMQ, OpenSearch, Novu, New Relic, Sentry, etc.
    - External Service Integrations: Clients/SDKs for Paystack, Flutterwave, Google, Cloudflare R2, etc.

4. Architectural Views

4.1. Logical View / Module View
    4.1.1. Overview
        The application is structured as a set of NestJS modules. A root `AppModule` imports core infrastructure modules and feature-specific modules.

        ```mermaid
        graph LR
            subgraph User Interaction
                UI(Frontend)
            end

            subgraph Backend API (NestJS)
                subgraph Core Modules
                    AppMod(AppModule) --> Cfg(ConfigModule)
                    AppMod --> DB(DatabaseModule)
                    AppMod --> Auth(AuthModule)
                    AppMod --> Redis(RedisModule)
                    AppMod --> Bull(BullMQModule)
                    AppMod --> Elastic(ElasticModule)
                    AppMod --> Notify(NotificationModule)
                    AppMod --> Common(Common Utils)
                end

                subgraph Feature Modules
                    Market(MarketplaceModule) -- uses --> Elastic
                    Consult(ConsultationModule) -- uses --> User(UsersModule)
                    Consult -- uses --> Timeslot(TimeslotService)
                    Consult -- triggers --> Pay(PaymentModule)
                    Cart(CartModule) -- holds --> ConsultEntity(Consultation Entity)
                    Cart -- holds --> CourseEntity(Course Entity)
                    Pay -- uses --> User
                    Pay -- uses --> PayGW(PaystackModule)
                    Pay -- updates --> Wallet(WalletService)
                    Pay -- triggers --> Enroll(EnrollmentModule)
                    Pay -- triggers --> Notify
                end
            end

            UI --> Market
            UI --> Consult
            UI --> Cart
            Market --> Consult
            Consult --> Pay
            Cart --> Pay # Assumed Checkout Flow
        ```

    4.1.2. Core Infrastructure Modules
        - `ConfigModule`: Manages environment variables and application configuration with Joi validation.
        - `DatabaseModule` (custom, likely wraps `TypeOrmModule`): Manages PostgreSQL database connections and entity registration.
        - `AuthModule`: Handles authentication (JWT, local, Google), authorization strategies, and user session management.
        - `CryptoModule`: Provides cryptographic functions (password hashing, JWT signing/verification).
        - `RedisModule`: Manages Redis connections for caching and potentially other uses (e.g., Throttler storage).
        - `BullmqCustomModule`: Manages BullMQ for background job processing.
        - `ElasticModule`: Integrates with Elasticsearch for search capabilities.
        - `NotificationModule` & `NovuModule`: Manages notifications via Novu (workflows, subscribers).
        - `SentryModule`: Integrates Sentry for error tracking and performance monitoring.
        - `MailerModule`: Handles email sending via SMTP with Handlebars templating.
        - `ClickHouseModule`: Integrates ClickHouse for analytics.
        - `StorageModule` (with `SimpleStorageService`): Manages file storage (e.g., S3).
        - `LoggerModule` (nestjs-pino) / Winston: Provides structured logging capabilities.
        - `TerminusModule`: For health checks.
        - `ThrottlerModule`: For API rate limiting.
        - `EventEmitterModule`: For in-application event handling.
    4.1.3. Key Feature Modules (Initial list based on analysis)
        - `UsersModule`: Manages user profiles, roles, and basic user data.
        - `CourseModule`: Manages course creation, content (sections, subsections, items), instructors, categories, levels, pricing, and related metadata.
        - `ConsultationModule`: Manages consultation offerings, consultant availability (timeslots), user bookings, and individual consultation sessions. It supports different consultation types (e.g., one-off, recurring) and integrates with `ElasticModule` for search, `PaymentModule` for billing, `NotificationModule` for reminders, and `ReviewsModule` for feedback. It has its own set of entities for consultations, bookings, timeslots, and type-specific metadata.
        - `PaymentModule`: Orchestrates financial transactions. It integrates with payment gateways (e.g., `PaystackModule`) to process incoming payments for orders. It manages `Payment` entities and includes a `WalletService` and `WalletController` for user fund management and withdrawals. It interacts closely with `OrderModule`, `UsersModule`, `CourseModule`, `ConsultationModule`, and `EnrollmentModule`.
        - `PaystackModule`: (If not already listed separately) Provides a dedicated interface for interacting with the Paystack payment gateway API, including payment initiation and webhook handling.
        - `InstructorsModule`: Manages instructor-specific profiles, linking them to base `User` accounts. It handles details like professional experience, achievements, expertise, and resume. It's tightly coupled with `UsersModule` and `CourseModule` (as instructors create courses) and interacts with `FilesModule` for resume/profile picture storage and potentially `EnrollmentModule` for viewing course enrollments. It may use a shared `ProviderService` for managing common profile sections like experiences and achievements.
        - `ConsultantModule`: Manages consultant-specific profiles, linking them to base `User` accounts. It handles details like professional experience, achievements, expertise, resume, and availability. It's tightly coupled with `UsersModule` and `ConsultationModule` (as consultants offer consultations) and interacts with `FilesModule` for resume/profile picture storage, `ReviewsModule` for feedback, and uses a shared `ProviderService` for managing common profile sections and availability.
        - `CartModule`: Provides shopping cart functionality, allowing users to add and manage items (courses, consultations) before purchase. It interacts with product modules for item details and `OrderModule` for checkout.
        - `OrderModule`: Manages the order creation process. It takes items (often from a cart), applies coupons, calculates totals, and creates `Order` records. It's a crucial intermediary to `PaymentModule` for initiating transactions and `EnrollmentModule` for post-payment fulfillment.
        - `ReviewsModule`: Handles user-submitted reviews and ratings for products (courses, consultations). Interacts with product modules to update aggregate ratings and `EnrollmentModule` for review eligibility. May integrate with `ElasticModule`.
        - `MarketplaceModule`: Powers the public-facing discovery of products. Relies heavily on `ElasticModule` for search/filtering and integrates `RecommendationModule`.
        - `StudentsModule`: Manages student-specific profiles (linked to `User`) and their interactions, particularly closely tied to `EnrollmentModule`.
        - `ImpressionModule`: Tracks user views/interactions with content like courses and consultations, likely feeding data to analytics or recommendation systems.
        - `EnrollmentModule`: Manages user enrollment in products, often triggered by successful payments. Includes an `EnrollmentListener` for event-driven enrollment creation and tracks granular progress.
        - `CommsModule`: Integrates with third-party communication platforms (e.g., Stream, CometChat) for features like chat/calls, managing user tokens for these services.
        - `CategoryModule`: Manages the hierarchical categorization of content (courses, consultations).
        - (Others to be detailed as analysis progresses: Product, etc.)
4.2. Development View
    4.2.1. Source Code Organization: Primarily within `src/`, with modules in dedicated subdirectories (e.g., `src/users`, `src/course`). Common utilities might reside in `src/common` or `src/shared`. Configuration in `src/config.ts` and `env/` directory.
    4.2.2. Key Frameworks and Libraries: NestJS, TypeORM, Express.js (underlying NestJS), Passport.js, Joi, BullMQ, Elasticsearch client, Novu SDK, Sentry SDK, Paystack SDK.
    4.2.3. Build Process: `npm run build` (uses `nest build`).
4.3. Deployment View
    4.3.1. Target Environment(s): Local, Development, Staging, Production (as indicated by `.env` file structure and `NODE_ENV` usage).
    4.3.2. Physical Infrastructure (Conceptual):
        - Application Server(s) running Node.js (v22 specified).
        - PostgreSQL Database Server.
        - Redis Server.
        - Elasticsearch Cluster.
        - ClickHouse Server.
        - InstantDB/Triplit Server (can be run via `docker-compose.yml` locally).
        - Dependencies on external services (Paystack, Novu, Sentry, Google, S3, Shlink).
    4.3.3. Containerization Strategy: `docker-compose.yml` provided is for `triplit-server`. The main application's containerization strategy for deployment is not yet fully detailed from current files but likely involves Docker.
    4.3.4. Technology Stack Summary: Node.js (v22), TypeScript, NestJS, PostgreSQL, Redis, Elasticsearch, BullMQ, Novu, Sentry, Paystack, S3, ClickHouse, InstantDB.

5. Data Architecture

5.1. Data Persistence Strategy
    - Primary Relational Database: PostgreSQL (managed via TypeORM).
    - Caching/Session/Rate-Limiting: Redis.
    - Search Indexing: Elasticsearch.
    - Analytics: ClickHouse.
    - Real-time/Collaborative Data: InstantDB/Triplit.
5.2. High-Level Data Model (Key entities identified so far)
    - `User`: Core user information, roles, credentials.
    - `Login`: Tracks login sessions/attempts.
    - `ForgotPassword`, `VerifyEmail`: Tokens for account recovery/verification.
    - `Course`: Main course details, pricing, instructor, category, level.
    - `CourseSection`, `CourseSubSection`, `CourseItem`: Hierarchical course content.
    - `Instructor`: Profile specific to instructors, linked to a `User`, contains professional details, resume, and courses they manage.
    - `Consultant`: Profile specific to consultants, linked to a `User`, contains professional details, resume, availability, and consultations they offer.
    - `Student`: Role-specific profiles linked to Users.
    - `Consultation`: Main consultation offering details, type, consultant, pricing.
    - `Booking`: User bookings for consultations.
    - `ConsultationTimeSlot`: Consultant availability.
    - `ConsultationSession`: Individual consultation sessions.
    - `OneOffConsultationMeta`, `RecurringConsultation`: Metadata for specific consultation types.
    - `Payment`: Records of financial transactions, linked to orders, users, and gateways.
    - `Withdrawal`: Records of user withdrawal requests from their wallets.
    - `Order`: Represents a user's intention to purchase items (courses, consultations).
    - `OrderItem`: Represents individual items within an order.
    - `Cart`: Represents a user's shopping cart.
    - `CartItem`: Represents individual items within a cart.
    - `Coupon`, `CouponUsage`: For managing discount codes.
    - `Category`: For course categorization.
    - `Student`: Profile for student users.
    - `Enrollment`, `SectionEnrollment`: Tracks user enrollment and progress in products.
    - `Review`: Stores user reviews and ratings.
    - `UserAuthTokens` (Comms): Stores tokens for external communication services.
    - `File`: For storing references to uploaded files (profile photos, course materials).
    - (Others to be added: PaymentTransaction, EnrollmentRecord, Product, etc.)
5.3. Data Flow for Key Use Cases
    - User Registration: Client -> API (`/auth/register`) -> `AuthService` -> `UsersService` (creates User) -> Role-specific service (creates Instructor/Student/Consultant profile) -> DB. Event emitted, Novu notification triggered.
    - Course Creation: Client -> API (`/course`) -> `CourseController` -> `CourseService` -> DB (creates Course, links to Instructor, Category, Files, etc.). Event emitted.
    - Consultation Booking: User (Client) browses consultations (via API, potentially using Elasticsearch) -> Selects a consultation and an available timeslot -> Submits booking request (API: `POST /consultation/booking`) -> `ConsultationController` -> `ConsultationService` (validates timeslot, creates `Booking` entity, potentially links to `ConsultationSession`) -> Triggers payment flow (via `PaymentModule`) -> On success, confirms booking, sends notifications (via `NovuModule`).
    - Checkout Flow: User adds items to Cart (via `CartModule`) -> User proceeds to checkout -> `OrderService` creates an `Order` from Cart items, applies coupons, calculates total, sets status to PENDING -> (Rest of payment flow as previously described, starting with `PaymentsService._initiatePaymentForOrder`).
    - Product Discovery: User searches/filters on Marketplace UI -> API (`/marketplace/*`) -> `MarketplaceController` -> `MarketplaceService` -> `ElasticModule` (queries Elasticsearch) -> Returns product list.
    - Leaving a Review: User (Student) submits review for a Course -> API (`/review`) -> `ReviewsController` -> `ReviewsService` (validates eligibility, e.g., via `EnrollmentService`) -> Saves `Review` entity -> Updates aggregate rating on `Course` entity.

6. Cross-Cutting Concerns

6.1. Security Architecture
    - Authentication: JWT-based. `LocalStrategy` for email/password, `GoogleStrategy` for Google OAuth. Managed by `AuthModule` and Passport.js.
    - Authorization: Role-based access control (RBAC) implied by `UserRole` and potentially fine-grained with `CaslGuard` (seen in `CourseController`).
    - Input Validation: `class-validator` DTOs with `ValidationPipe`.
    - HTTP Security Headers: `helmet` middleware.
    - CORS: Enabled via `app.enableCors()`.
    - Secret Management: Primarily through environment variables (`.env` files, `ConfigModule`). `KEYS_DIR` in config suggests potential for local key storage.
6.2. Error Handling and Fault Tolerance
    - Global Exception Filter: `AllExceptionsFilter` in `main.ts` for standardized error responses.
    - Custom exceptions defined per module (e.g., `PhotoNotFoundException` in `UsersModule`).
    - Sentry: Integrated for error tracking and reporting (`SentryModule`, process-level handlers in `main.ts`).
6.3. Logging and Monitoring
    - Structured Logging: `nestjs-pino` or Winston (conditional in `main.ts`). `AppRequestLoggerMiddleware` for HTTP request logging.
    - Performance Monitoring: Sentry APM (implied by `instrument.ts` and Sentry setup).
    - Health Checks: `TerminusModule` imported, likely for `/health` endpoints.
6.4. Configuration Management
    - `@nestjs/config` loading environment-specific `.env` files from `env/`.
    - Joi schema validation (`src/config.ts`) for environment variables.
    - Type-safe access via `ConfigService` and `Config` interface.
6.5. API Design
    - RESTful principles generally followed.
    - Standardized response format via `ResponseInterceptor` (seen in `main.ts`).
    - API documentation via Swagger (setup in `main.ts`, available at `/docs`).
6.6. Asynchronous Processing
    - `BullmqCustomModule` for background jobs (e.g., email sending, long computations).

7. Design Rationale

7.1. Why NestJS?
    - Opinionated framework providing structure for large applications.
    - Excellent TypeScript support.
    - Modular architecture promotes separation of concerns.
    - Built-in support for microservices, testing, and common patterns.
7.2. Why a Modular Architecture?
    - Improved maintainability and testability.
    - Allows for parallel development by teams.
    - Clear boundaries between different functional areas of the application.
7.3. Rationale for Database Choices
    - PostgreSQL: Robust, feature-rich relational database suitable for complex data models.
    - Redis: High-performance key-value store ideal for caching, session management, and rate limiting.
    - Elasticsearch: Powerful search engine for providing fast and relevant search results (e.g., for courses).
    - ClickHouse: Columnar database optimized for OLAP queries and analytics.
    - InstantDB/Triplit: Real-time database capabilities.

Appendix

- (Diagrams can be added here later if text-based representations are feasible)