In the case of a failure, your Ingress Replicator(s) or Search Service nodes may be
unreachable. This topic describes what happens during an outage.
Note: To avoid non-recoverable disk failures, Jive Software recommends that you configure
the Ingress Replicator journals and Search Service indexes so that they are written to
durable storage. For each Ingress Replicator, allocate at least 20GB for journal
storage. For each Search Service, allocate at least 50GB for index storage. Monitor
these storage volumes for remaining capacity, maintaining 25% free capacity.
In the case of a failure of any given node in your HA Search configuration,
here's what happens:
- Ingress Replicator node fails
- The Ingress
Replicator journals everything to disk to guarantee all ingressed activities
will be delivered at least once. If the service fails or is stopped, it will
send any remaining journaled events when it starts back up. If the service
cannot come back up due to a non-recoverable disk failure, then a full rebuild
will be required (see
Rebuilding an On-Premise HA Search Service).
If both Ingress Replicators die (or you have only one and it dies), for the
duration of the outage no new content will be indexed; but, when the Ingress
Replicator comes back online, the Search Service will catch up with the indexed
content (due to local caching on the web app nodes); therefore, the Search
Service will not have missed anything.
- Search service node fails)
- If Search Service
1 or 2 is offline for any reason, the Ingress Replicator will retain
the undelivered activities. When Search Service 1 or 2 is restored to a healthy
state, the undelivered activities will be sent to the restored service. While
previously undelivered activities are being fed into the newly restored service,
the search indexes will be out of sync. After all undelivered activities have been received
by the restored service, the indexes will be in-sync. If the service cannot be
restored due to a non-recoverable disk failure, then you'll need to remove and re-add
the affected search service (see
Adding an On-Premise HA Search Service Node). If you leave a
Search Service down for a very long period of time (e.g., many weeks), you may run
out of disk space because the Ingress Replicator services will be persisting to disk until
the configured Search Service is restored. If you don't plan to restore the offline
Search Service, then remove the offline Search Service from all Ingress
Replicator configuration files and restart the Ingress Replicators.