Skip to content

Flask Architecture

The Flask app keeps HTTP concerns at the edge and integration/business logic in plain Python modules.

Request Boundary

  • app/blueprints/ owns Flask route functions, request parsing, decorators, and conversion to flask.Response.
  • app/http.py contains response adapters. Use response_from_result() to turn an EndpointResult into a Flask response.
  • app/error_handlers.py assigns a request id to every request, echoes it in X-Request-ID, and converts server-side failures into JSON error responses.
  • app/route_support.py and app/helpers/inbound_auth.py remain Flask-aware because they read request headers and can reject requests before a view runs.

Route Groups

Routes are registered in app/routes.py through Flask blueprints.

  • core_bp is always registered and owns lightweight operational routes such as health checks. public_root_bp is registered only when internal routes are disabled, so the public app still has a root response.
  • report_exports_bp, crm_reports_bp, and delivery_model_reports_bp own CSV and report-like endpoints used by Power BI or operators. Report endpoints use Basic auth, with the Delivery Model alert email preview as the current unauthenticated preview exception.
  • project_webhooks_bp owns external webhook entrypoints from Zoho CRM/Flow. These routes are token-protected and remain available even when internal routes are disabled.
  • delivery_model_automations_bp, projects_proxy_bp, and internal_debug_bp are internal-only groups. They are registered only when EXPOSE_INTERNAL_ROUTES is truthy.

Do not add new routes directly in server.py or app/__init__.py; put them in the nearest blueprint and let register_routes() wire the group into the app.

Config And Environment

Runtime configuration is centralized in app/settings.py.

  • Settings.from_env() loads .env when no explicit environment mapping is provided, trims string values, parses comma-separated email recipient lists, and parses boolean-style EXPOSE_INTERNAL_ROUTES.
  • get_settings() caches the settings object for normal runtime use. reset_settings_cache() exists for tests or tools that intentionally change environment variables mid-process.
  • Settings are validated by feature-specific methods at first use, not at app startup. This lets tests and narrow scripts exercise one feature without configuring every integration.

Required environment groups:

  • Zoho OAuth: CLIENT_ID, CLIENT_SECRET, REFRESH_TOKEN, ACCOUNTS_BASE.
  • Zoho CRM: API_DOMAIN.
  • Zoho Projects: PROJECTS_API_DOMAIN, PROJECTS_PORTAL_ID.
  • Report Basic auth: EXCEL_BASIC_USER, EXCEL_BASIC_PASS.
  • Inbound token auth: INBOUND_API_TOKEN.
  • CRM offer widget connection: OFFER_WIDGET_TOKEN (offer APIs only).

Optional or defaulted values:

  • ACCOUNTS_BASE defaults to https://accounts.zoho.com.au.
  • PROJECTS_PORTAL_LINK_BASE defaults to the Peppermint Projects portal link.
  • ALERT_RECIPIENT_EMAILS is optional and parsed as a comma-separated list.
  • DEBUG_RECIPIENT_EMAILS is optional and parsed as a comma-separated list. It is for standalone debug notifications sent through Zoho Mail, not CRM record emails.
  • EXPOSE_INTERNAL_ROUTES defaults to false; accepted truthy values are 1, true, yes, and on.
  • DASHBOARD_SESSION_SECRET is an optional strong random signing key for browser report sessions. Share a stable value across production workers. Without it, each app instance generates an ephemeral key; local restarts require signing in again. Never reuse the report password as this key.
  • DASHBOARD_COOKIE_SECURE=1 requires HTTPS for the report session cookie. Enable it on HTTPS deployments; leave it unset for local HTTP development. Reverse proxies must expose the correct request origin to the application.

Auth Model

  • Report routes use basic_auth_required() from app/route_support.py. Credentials come from EXCEL_BASIC_USER and EXCEL_BASIC_PASS, and failed requests receive a Basic auth challenge before report builders run.
  • The commercial dashboard, service-review endpoint and forecast-input endpoint additionally accept a signed browser session. The dashboard redirects unauthenticated visitors to /commercial-margin/sign-in, which checks those same report credentials through a normal form rather than a native browser dialog. Browser sessions last eight hours, are scoped to /commercial-margin, and use HttpOnly/SameSite=Strict cookies. Cookies contain no credentials or unkeyed password hashes. Credential changes invalidate existing sessions. Form sign-in/out use CSRF checks; redirects can only return to the dashboard, preserving its query filters. Expired report sessions return a sign-in URL without issuing a Basic challenge, allowing the dashboard to show the form. Basic clients remain supported; this cookie does not authorize other exports or automation routes. Tests: tests/test_dashboard_browser_auth.py.
  • Offer APIs additionally accept X-Offer-Token through offer_api_auth_required() in app/helpers/offer_auth.py. This separate shared credential authorizes only catalogue, draft, quote and selection operations. The standalone offer page retains report Basic auth. See widget setup.
  • Webhooks and automation/proxy routes use token_protected() from app/helpers/inbound_auth.py. Callers can send either Authorization: Bearer <token> or X-Pepperback-Token.
  • Missing auth configuration is treated as a server error. Missing or invalid caller credentials are rejected with 401.
  • Internal-only routes are not merely auth-protected; they are not registered at all unless EXPOSE_INTERNAL_ROUTES is enabled.

Error And Logging Contract

  • Every response receives X-Request-ID. If the caller sends X-Request-ID, the app preserves that value; otherwise the app generates one.
  • Server-side error responses include error, code, and request_id. Zoho upstream failures also include details with method, URL, status, and body.
  • Use app.logging.log_event() for application logs. It emits one JSON object per log line and includes request_id automatically when a request context exists.
  • Use info for lifecycle summaries, warning for recoverable partial data, retries, skipped records, and cache/recipient lookup issues, and error for failed writes or failed upstream calls.
  • A broad except Exception must either re-raise or call log_event(..., exc_info=True, ...) before returning an explicit error payload. Background jobs must log stack traces because no HTTP caller may see their failures.
  • Do not use print() or SystemExit in app/; tests enforce this so errors are visible in structured logs and request error responses.

Service Results

  • Service modules return app.results.EndpointResult for endpoint-like outputs.
  • Use json_result, csv_result, html_result, or bytes_result instead of constructing flask.Response in service code.
  • Service functions should use *_result names when their output is intended for a route adapter.

Naming Contract

  • A leading underscore means module-local only.
  • Helpers shared across modules must use public names and be imported explicitly from their owning module.
  • Do not import or alias a leading-underscore name from another app module. Tests enforce this for application code.

Integration Clients

  • app/zoho/ clients own Zoho URLs, auth headers, request execution, and raw upstream response handling.
  • ZohoBaseClient owns OAuth header construction, request timeouts, URL normalization, and the split between raw request_response() calls and parsed request_json() calls.
  • Parsed client methods raise ZohoAPIError for upstream 4xx/5xx responses. Methods ending in _response return the raw upstream response for proxy routes and services that must preserve status-code/body behavior.
  • ZohoCRMClient owns CRM module pagination, record fetch/update/create methods, and Delivery Model mail actions.
  • ZohoMailClient owns standalone debug email sends that must not be attached to CRM records.
  • ZohoProjectsClient owns Projects portal and REST API URL shapes, project task/tasklist/phase operations, timelog collection, team lookups, and selected fallback payload encodings required by Zoho Projects.
  • Blueprints may still proxy selected upstream Zoho responses directly with proxy_json_response() when the endpoint is intentionally a pass-through.
  • Business modules should depend on client methods or injected callables, not on Flask request/response objects.

Report Flow

Report endpoints should follow this shape:

  1. Blueprint route handles HTTP method/path and applies Basic auth.
  2. Route calls a builder/service function, injecting collectors where tests need isolation.
  3. Service code fetches CRM or Projects data through Zoho clients or collection helpers.
  4. Service returns an EndpointResult via csv_result, json_result, html_result, or bytes_result.
  5. Route converts the result with response_from_result().

Keep report builders deterministic where possible: normalize records into explicit schemas, keep CSV column order stable, and cover schema changes with characterization tests.

The commercial-margin labour export maps and writes one timelog row at a time, without materializing the expanded row list or a pandas export table. The shared timelog CSV writer also accepts existing list-based callers. It retains the column order, blank missing values, and numeric values; integer values are written without a decimal suffix even when other rows have missing values. The final CSV text and the source caches still reside in memory. Portal task and timelog collectors discard expired snapshots before refetching and release raw responses between pages/months, avoiding overlap with replacement data. Source caches have a five-minute lifetime, and the labour history starts in January 2025.

Webhook Flow

Webhook and automation endpoints should follow this shape:

  1. Blueprint route applies inbound token auth.
  2. Route reads a JSON object with json_object_payload() and returns 400 for non-object payloads or missing required identifiers.
  3. Route delegates to an automation or service module.
  4. Service code performs Zoho reads/writes through client methods or injected callables.
  5. Route returns either json_response() for direct payload/status tuples or response_from_result() for EndpointResult outputs.

Webhook services should return explicit partial-failure payloads when a workflow can continue after a failed sub-step, and should log unexpected failures through the structured logging contract.

Service Module Map

  • app/reporting_portal.py owns Zoho Projects portal collection, report pagination, short-lived cache state, and task metadata mapping.
  • app/reporting_metadata.py owns shared CRM/Projects metadata joins used by report exports and Delivery Model reports.
  • app/crm_reports/ owns CRM report fields, normalizers, MSP billing calculations, and result builders. app/crm_reporting.py is a compatibility facade.
  • app/delivery_models/alert_email.py owns Delivery Model alert email rendering and test email payloads.
  • app/delivery_models/alert_recipients.py owns alert recipient lookup and email deduplication.
  • app/delivery_models/timelog_mapping.py owns Delivery Model timelog scope matching and CSV column shape; timelog_monthly.py owns recurring monthly aggregation. timelogs.py is a compatibility facade.
  • app/automations/delivery_model_completion_calculations.py owns completion percentages, threshold crossing, and recurring current-month calculations. delivery_model_completion.py owns state-sync orchestration.
  • app/delivery_models/archived_backfill_cache.py owns archived timelog backfill cache, status, lock, and background generation behavior. archived_backfill_restapi.py owns archived Projects REST API collection and payload normalization. archived_backfill.py is the endpoint facade and report orchestration layer.

Facade modules preserve older imports for routes and tests. New code should prefer importing from the focused modules directly unless it is intentionally working at an endpoint orchestration boundary.

Test Strategy

  • Route-map tests preserve the public and internal route contracts, including EXPOSE_INTERNAL_ROUTES behavior.
  • Auth boundary tests verify Basic auth and inbound token rejection before builders or automations run.
  • Settings tests cover defaults, feature-scoped validation, list parsing, and cache reset behavior.
  • Zoho client tests use fake transports/responses to validate URL construction, headers, parsed response handling, and ZohoAPIError details without making network calls.
  • Report characterization tests assert CSV schemas, lookup flattening, billing calculations, and route registration/auth for report endpoints.
  • Webhook and automation tests exercise payload validation, delegation, idempotent setup behavior, and partial failure payloads.
  • Static enforcement tests keep app code from hiding failures with print() or SystemExit, and prevent cross-module imports or aliases of private underscore-prefixed names.