The FormWork logging system is a distributed and resilient system and serves two purposes:
- monitor the performance of the system
- be alerted when an exception occurrs
The infrastructure components are shown in the following image:
How the distributed logging works
In logging system there are sources emitting logs, receptors that are enabled to receive and storage saving them.
Sources are three: the Excel client, the Administration Console and the FormWork server. When one of these sources generates a log, because of a significant event as the beginning of an operation, the log is received by the FormWork instance and stored in local storage ('Instance Log Buffer' in the figure above). Clients Excel log simultaneously on the event viewer.
The logs in this buffer are cleaned and transformed using ETL, because the clients may not know all the information necessary to build an acceptable entry log.
For example, an Excel client might know that it launched the procedures no. 125 but it could not know its name, so the scheduled task integrates this information, and deletes false positives and duplicate entries. After the execution of this task, a portion of log accumulated in the buffer instance are clean and ready for publication. The server now elects one of the clients to the role of publishers and sends a packet of compressed logs and the 'coordinates' on where these logs are shipped.
A background thread of the client receives the packet and try to send it to the logging service on cloud. If this action is successful, the thread notifies the FormWork instance which mark the shipped logs.
The next ETL and routing task will start from the log next to the last in the package.
Cloud, the log service that received the log package sends it to the buffer and the buffer saves it.
Within the ICONSULTING network the process is always active, periodically dumps a portion of the compressed log, decompresses and decrypts it, and sends it to the syslog server.
Upon completion the process marks the logs as 'received' and then they can be overwritten
Exceptions are directly sent to the exception handler. If the client cannot reach Internet they are saved to a local cache found in C: \ ProgramData \ IsolatedStorage and sent when connectivity is restored.