Incident Report on clearing server outage and market halts in Genium INET on November 23rd 2015. (50/15)


Description of the Incident

On Monday November 23rd at 09:01 (09:00:41) CET both the primary and secondary clearing server process failed. As a result no trade confirmation broadcasts (BD6/Trade Capture reports) and no trade statistics (BD2) were sent from the Genium INET system. Matching confirmation from the matching engine was available in the Firm Order Book broadcast (BO5) and in the FIX Execution reports.

Affected Markets are specified below.

After confirmation that an immediate resolution and restart of the clearing servers was not possible, Nasdaq Stockholm AB (“the exchange”) decided to set the Affected Markets in Trading Halt. This was done at 09:16 (09:16:02 - 09:16:04).

After identifying the root cause and a solution, the clearing servers were successfully restarted at 09:27. The trades that had been made between 09:01 and 09:16 and which had been queued to the clearing servers were disseminated.

The Affected Markets were scheduled for resumption accordingly;
For Swedish Stock (including Flexible), Finnish Stock on Request, Euro Index, Danish Stock (including Flexible), Norwegian Stock (including Flexible) Derivatives markets; the PREOPEN (Order Management) was set to start at 09:50 and OPEN (continuous trading) at 10:10 CET.
For the Swedish Index (including Flexible), Danish Index (including Flexible), Norwegian Index (including Flexible), World Index Derivatives markets; the PREOPEN (Order Management) was set to start at 09:50, the CLIN (Call Interaction) was set to start at 10:00 CET, followed by Uncross and OPEN (continuous trading) at 10:10 CET.
The Danish Bond, Danish TMC Bond, Euro Bond, Euro TMC Bond, Norwegian Bond, Norwegian TMC Bond, Swedish Bond, Swedish TMC Bond Derivative markets were set to go into OPEN (continuous trading) at 10:10.

The disturbance messages sent regarding the incident can be found on: http://www.nasdaqomx.com/transactions/technicalinformation/itnotices. Filter on “Production System Disturbances” and date 2015-11-23. Seven messages were sent all in all.

 

Affected Markets

DANISH INDEX
DANISH STOCK
DANISH FLEXIBLE INDEX
DANISH FLEXIBLE STOCK
DANISH BOND
DANISH TMC BOND
EURO BOND
EURO TMC BOND
EURO INDEX
EURO FLEXIBLE INDEX
FINNISH STOCK
FINNISH STOCK ON REQUEST
FINNISH FLEXIBLE STOCK
NORWEGIAN BOND
NORWEGIAN TMC BOND
NORWEGIAN INDEX
NASDAQ OMX NORWEGIAN STOCK
NORWEGIAN STOCK
NORWEGIAN FLEXIBLE INDEX
NORWEGIAN FLEXIBLE STOCK
SWEDISH BOND
SWEDISH INDEX
SWEDISH TMC BOND
SWEDISH FLEXIBLE INDEX
SWEDISH FLEXIBLE STOCK
SWEDISH STOCK
WORLD INDEX SEK

 

Root Cause

The root cause was a software defect in the clearing server.
 

Plan for future prevention of the problem

The root cause has been identified and the software defect corrected.

Finally we apologize for the inconvenience this incident may have caused you.

Stockholm, November 25th, 2015

 

Gunilla Hellqvist                                                                           Fredrik Maartmann
Senior Vice President,                                                               
Enterprise Risk Manager
Global Market Operations                                                          
Global Market Operations


Attachments

External Incident Report -  Market halts Nov 23 2015.pdf