Skip to main content

    Downtime Notifications

    Downtime Notications are a mechanism for handling outages of external services in the consuming front-end applications.

    Overview

    • Available as a React component in vets-website.
    • Consumers of the React component register required-services as dependencies via React props
    • Driven by PagerDuty maintenance windows so that downtime for a certain service can be scheduled without requiring a deployment. More info on PagerDuty maintenance windows.
    • After a maintenance window is added in Pager Duty, an application wrapped in the Downtime Notification React component typically responds in one of three ways:
      1. Rendering the application as normal if the maintenance window does not start for upwards of an hour.
      2. Displaying a dismissible modal informing the user of upcoming downtime and rendering the application as usual if the maintenance window starts within the hour.
      3. Rendering an alert banner informing the user of active downtime instead of rendering the application if the maintenance window has already started.

    How to use

    Prerequisites

    • Identify required-services for the application
      • These services should be available in externalServices if they have not already been. The value must match that in the devops config.
      • These dependencies may not be directly accessed by your application, but are more likely to be dependencies of the API method(s) consumed by your application. For example, the Search app requires Search.gov to function, even though it doesn't directly hit Search.gov.

    Forms library applications

    1. Import the services enum
    import { externalServices } from 'platform/monitoring/DowntimeNotification'
    1. Add a downtime property and dependencies array to the root of the form config. For example:
     downtime: {
    dependencies: [externalServices.evss, externalServices.emis, externalServices.mvi, externalServices.vet360],
  },
    1. Add additional properties to downtime as needed:
    • message
      • a react component displayed in place of the default downtime message
      • custom downtime messages are discouraged
    • requiredForPrefill
      • true:
        • given a user is logged in, when any services in the dependencies array are down, then display downtime notification before starting a form
        • given a user is logged in and has a saved form, when any services in the dependency array are down, then display the downtime notification before application submission
        • given a user is not logged in,
      • false [default]: when any services in the dependencies array are down, then display the downtime notification before application submission
    1. If you need your FormBuilder app to have downtime notifications on intro page you also need to follow React implimentation below
      • The above implimentation only adds downtime notifications to the review page. Most forms will probably require to also the notifications on intro page so users dont find out at the end of the form.

    The DowntimeNotifications component wraps the FormStartControls on the introduction page and the SubmitController on the review page. Users can edit and save forms for later during a downtime unless requiredForPrefill is true.

    React applications

    1. Import the Downtime Notification React component into the application React code, usually the topmost container.
    2. In the render method, render a Downtime Notification React component passing the identified dependencies as props and the affected components as React children.

    Example

    Suppose an application My App leverages a service called Fancy Service, and Fancy Service was known to be offline for maintenance every now and then. During those timeframes, it is desirable to show a message to the user explaining that My App is unavailable at this time so that users aren't given the impression that they are running into unexpected errors and so that Fancy Service isn't needlessly hit. This should also be easy for engineers to do, without requiring any deployments to publish/unpublish the messaging. This is a pretty typical use case for the DowntimeNotification React component.

    React code

    First, the topmost React container of My App is modified to import and consume the Downtime Notification React component.

    Note that MyAppDataGrid would issue the API request to Fancy Service during componentDidMount, rather than MyApp. This is because MyAppDataGrid is passed a child component to DowntimeNotification. DowntimeNotification will fetch the currently-registered maintenance windows from the VA API (which in return reads the maintenance windows from PagerDuty), and if it discovers active downtime for fancyService, then the componentDidMount lifecycle method of MyAppDataGrid will not execute. This way, API requests to Fancy Service won't continue to send during downtime.

    src/applications/my-app/containers/MyApp.jsx

    import React from "react";
import { connect } from "react-redux";

import {
  DowntimeNotification,
  externalServices,
} from "platform/monitoring/DowntimeNotifications";

import { fetchFromFancyService } from "../actions";

import MyAppDataGrid from "../components/MyAppDataGrid";

class MyApp extends React.component {
  render() {
    return (
      <>
        <h1>My App</h1>
        <DowntimeNotification
          appTitle="my app"
          dependencies={[externalServices.fancyService]}
        >
          <MyAppDataGrid
            fetchFromFancyService={this.props.fetchFromFancyService}
          />
        </DowntimeNotification>
      </>
    );
  }
}

const mapStateToProps = (state) => store.myApp;

const mapDispatchToProps = {
  fetchFromFancyService,
};

export default connect(mapStateToProps, mapDispatchToProps)(MyApp);

    The render-flow for this container is:

    1. MyApp's render method is executed
    2. DowntimeNotification issues a GET request to the VA API for all registered maintenance windows.
    3. DowntimeNotification processes the response JSON data into a map.
    4. DowntimeNotification searches the downtime map for a service called fancyService, as passed in its dependencies prop.
    5. If downtime for fancyService is found:
      • If the current time is in the timeframe of the downtime window, render an alert banner informing the user that this application is undergoing maintenance.
      • Otherwise, DowntimeNotification will renders its children, in this case MyAppDataGrid. A dismissible modal is also rendered if downtime is approaching within the hour.

    What the notification looks like one hour before service goes down with no custom content

    downtime one hour before

    This is what the downtime notification looks like during downtime period

    downtime during downtime period

    Adding a new PagerDuty service

    If you've created a new backend service that a React application depends on, follow these steps to allow the React application to fetch its downtime windows:

    Configuration

    • Login to PagerDuty (https://dsva.pagerduty.com/)
    • Create your PagerDuty Service (https://dsva.pagerduty.com/service-directory)
      • This may require you to create a PagerDuty Person, Escalation Policy, and/or Team that is specific to this service.
      • Follow the naming convention of other backend services in PagerDuty by prepending "External: " to the service name (i.e. "External: MyService").
      • You may also want to create multiple services for multiple enviornements (i.e. "External: MyService", "Staging: External: MyService", "Dev: External: MyService").
    • In the vets-api repo, add your service to the list of maintenance services in config/settings.yml.
    • In the devops repo, make the same configuration change to each enviornemnt's config file.
    • Deploy all changes.

    Testing

    Other examples

    1. The Search app is a basic example of an application having a single service dependency, in this case being Search.gov. The Facility Locator is another example.
    2. The Account app is another simple example. This application has two dependencies. If either is down, then the application is considered in maintenance.
    3. The Sign-In Modal is an example of a component that renders messaging about service downtime, but it does not affect the functionality of the component. Instead, it's more a heads-up for the user about potential difficulty. This is also the case for Letters.
    4. The Dashboard app consists of components with their own unique dependencies, so that a certain service being in maintenance should result in reduced but partial functionality for the Dashboard. It serves as an example of a more complex implementation of Downtime Notifications.