MQSeries

MQSeries :

 Introduction

MQSeries is an IBM supplied Business Integration software (Middle ware).

It is a Messaging and Queuing system provided for program-to-program communication.

Programs within an application communicate by writing and retrieving application-specific data (messages) to/from queues, without having a private, dedicated, logical connection to link them.

Messaging means that programs communicate with each other by sending data in messages and not by calling each other directly.

Queuing means that programs communicate through queues. Programs communicating through queues need not be executed concurrently.

MQSeries is used in a client/server or distributed environment. Programs belonging to an application can run in one workstation or in different machines on different platforms.

It allows business applications to integrate and communicate across a spectrum of desktop and mainframe systems, overcoming inconsistencies with different network protocols and all major commercial platforms.

Key benefits of MQSeries:

·         A single, multi-platform API

·         Isolation from communications programming

·         Time-independent processing

·         Assured message delivery, even when programs or networks fail

·         Robust middleware for high-performance distributed application integration.

Basics of MQSeries.

Message

A message consists of two parts.

·         Data that is sent from one program to another.

·         Message descriptor that identifies the message (message ID) and contains control information.

MQSeries messages are of four types.

·         Datagram - A message containing information for which no response is expected.

·         Request - A message for which a reply is requested.

·         Reply - A reply to a request message.

·         Report - A message that describes an event such as the occurrence of an error.

MQSeries differentiates between persistent and non-persistent messages.

·         Delivery of persistent messages is assured. They are written to logs and survive system failures.

·         Non-persistent messages cannot be recovered after a system restart.

Queue Manager

The heart of MQSeries is the message queue manager (MQM).  A queue manager is a system program that provides queuing services to applications. It manages Queues of messages.

·         It provides an application programming interface so that programs can put messages on, and get messages from, queues.

For example, the MQPUT API puts a message on a queue to be read by another program using the MQGET API.

·         It provides additional functions so that administrators can create new queues, alter the properties of existing queues, and control the operation of the queue manager.

A program may send messages to another program that runs in the same machine as the queue manager, or to a program that runs in a remote system, such as a server or a host. The remote system has its own queue manager with its own queues.

Queues

Queues are objects belonging to a queue manager. Message queues are used to store messages sent by programs.

·         Local Queue - A queue is local if it is owned by the queue manager to which the application program is connected.

·         Remote Queue- A queue is remote if it is owned by a different queue manager.

·         Transmission Queue - A remote queue is associated with a transmission queue. Transmission queues are local and used as an intermediate step when sending messages to remote queues.

·         Dynamic queues – They are local queues. They are often used in conversational applications, to store intermediate results.

·         Model Queue - A model queue is not a real queue. It is a collection of attributes that are used when a dynamic queue is created.

·         Alias Queue - Alias queues are not real queues but definitions. They are used to assign different names to the same physical queue. This allows multiple programs to work with the same queue, accessing it under different names and with different definitions.

·         Initiation Queue - An initiation queue is a local queue to which the queue manager writes a trigger message when certain conditions are met on another local queue.

·         Reply-To Queue - A request message must contain the name of the queue into which the responding program must put the reply message.

·         Dead-Letter Queue - A queue manager must be able to handle situations when it cannot deliver a message.

Channels

Messages are transmitted via channels. A channel is a logical communication link. In MQSeries, there are two different kinds of channels:

·         Message channel  - An MCA is a program that transfers messages from a transmission queue to a communication link or vice versa.

·         MQI channel - A Message Queue Interface (MQI) channel connects an MQI client to a queue manager in a server machine. MQI clients don’t have a Queue Manager of their own. An MQI channel is bidirectional.

Process Definitions.

A Process Definition object defines an application to a queue manager. For example, it contains the name of the program to trigger when a message arrives.

Working of MQSeries.

The queuing system consists of the following parts:

·         Queue Manager (MQM)

·         Listener

·         Trigger Monitor

·         Channel Initiator

·         Message Channel Agent (MCA)

The working of MQSeries is illustrated in the following steps.

1)    When the application program wants to put a message on a queue it issues an MQPUT API. This invokes the MQI(Message Queuing Interface).

2)    The queue manager checks if the queue referenced in the MQPUT is local or remote.

3)    If it is a remote queue, the message is placed into the transmission (xmit) queue.

4)    Each remote queue must be associated with an transmit queue. Usually, all messages destined for one remote machine use the same transmit queue.

a)    The queue manager adds a header that contains information from the remote queue definition, such as destination queue manager name and destination queue name.

5)    Transmission is done via channels. Channels can be started manually or automatically.

a)    To start a channel automatically, the transmit queue must be associated with a channel initiation queue.

b)    The channel initiator is an MQSeries program that must be running in order to monitor initiation queues.

c)    When the channel initiator detects a message in the initiation queue, it starts the message channel agent (MCA) for the particular channel.

d)    This program moves the message over the network to the receiving end.

e)    On the receiving end, a listener program must have been started.

f)    The listener monitors a specified port, by default the port dedicated to MQSeries is 1414.

g)    When a message arrives, it starts the message channel agent. The MCA moves the message into the specified local queue .

h)    The program that processes the incoming message can be started manually or automatically.

i)     To start the program automatically, an initiation queue and a process must be associated with the local queue, and the trigger monitor must be running.

j)     The MCA puts the incoming message into the local queue and a trigger message into the initiation queue.

k)    The initiation  queue is monitored by the trigger monitor. This program invokes the application program .

l)     The application issues an MQGET API to retrieve the message from the local queue.

 

MQSeries for AS/400

The AS/400 operating system, OS/400, already has two queue systems

• Message queues (MSGQ)

Message queues are used for system and program messages for the operator.

• Data queues (DTAQ).

Data queues are used to exchange data between applications in one or more AS/400 systems, and between an AS/400 and clients (PCs)

The Reasons for using MQSeries on AS/400 despite using these queuing systems are,

• MQSeries for AS/400 can store queue data persistently. AS/400 standard data queues cannot.

• MQSeries for AS/400 can synchronize transactions and AS/400 databases with commit/rollback. AS/400 standard data queues cannot.

• MQSeries enables application programs running on different platforms to communicate with each other using the Message Queuing Interface (MQI) which is consistent over all platforms.

• Channel sequence checking between systems assures that no data is lost (not implemented for AS/400 standard remote data queues).

• MQSeries allows remote administration across platforms.

• MQSeries for AS/400 can browse a message (non-destructive read). AS/400 data queues cannot.

• MQSeries enables application programs running on different platforms to communicate with each other using the Message Queuing Interface (MQI) which is consistent over all platforms.

MQSeries is not just another AS/400 Queuing System.  These are the key advantages of MQSeries over AS/400 Data Queues and Message Queues.

MQSeries objects for AS/400.

AS/400 MQ objects are known to the OS/400 operating system as object type  *USRSPC (user space) in the QMQMDATA library.

MQSeries queues, channels, and other objects are stored in the AS/400

object type dataspace.

MQSeries Queue Manager.

The AS/400 allows only one queue manager instance.

To Create a Queue manager one should log in as security officer (QSECOFR) or as a MQ administrator  (MQADM).

The following MQ CL Command creates a Queue Manager:

CRTMQM    MQMNAME(MYQMGR)

The CRTMQM panel allows the entry for various parameters in the command.

Every Queue manager needs some default definitions for Queue manager objects.

This is done by calling the program   “amqsdef4”

CALL qmqm/amqsdef4

To start the MQM and display its attributes the following commands are used,

STRMQM

DSPMQM

MQSeries client

An MQSeries client is an independently installable component of an MQSeries  product. It allows to run MQSeries applications,  by means of a communications protocol, to interact with one or more MQI servers on other platforms and to connect to their queue managers.

MQSeries clients do not have a queue manager of their own. Client machines connect to a queue manager in a server. This MQM manages the queues for all clients attached to it.

AS/400 only can be only a MQSeries server and it cannot be a MQSeries client.

The MQSeries Clients communicate with the Queue Manager in a server machine via a MQI  channel.

A Queue Manager  communicates with another Queue Manager via a Message Channel Agent ( MCA).

Manipulating MQM objects

In Non AS/400 Systems the Utility RUNMQSC is used to manipulate the  Queue Manager objects.

In AS/400 System the MQSC commands cannot be run interactively.

To run the MQM commands interactively AS/400 has separate set of CL commands.

All the MQ CL commands is provided in the menu CMDMQM.

GO CMDMQM provides a menu to execute the MQ CL commands interactively.

The MQSC commands can be run in AS/400 system in batch from a file member in a source file .  The CL command used is,

STRMQMMQSC SRCMBR(MYCOMA) SRCFILE(MYLIB/TST)

Communication between Queue Managers

Message Channels are used for communication between queue managers in residing in two separate systems.

Each machine has a queue manager installed and each queue manager

manages several local queues.

Messages destined for a remote queue manager are put into a remote queue.

A remote queue is not a real queue; it is the definition of a local queue in the remote machine. A remote queue is associated with a transmission (xmit) queue which is a local queue. Usually, there is one xmit queue for each remote queue manager.

Each xmit queue is associated with a message channel. The message channels are unidirectional.

Therefore two message channels are to be defined for  bidirectional  messaging.

The required objects for connecting two Queue Managers are,

·         A remote queue definition that mirrors the local queue in the receiver machine and links to a transmission queue.

·         A transmission queue that holds all messages destined for the remote system until the channel transmits them (B in system A and A in system B)

·         A sender channel that gets messages from the xmit queue and transmits them to the other system using the existing network (A.TO.B in system A and B.TO.A in system B)

·         A receiver channel that receives messages and puts them into a local queue (B.TO.A in system A and A.TO.B in system B)

·         A local queue from which the program gets its messages (Q2 in system A and Q1 in system B)

	System A	System B
Queue1	DEFINE QREMOTE(Queue1)   REPLACE + RNAME(Queue1)  + RQMNAME(SYSTEMB) + XMITQ(B) + DESCR(‘Queue 1 in system B’ )	CRTMQMQ QNAME(‘Queue1’)  QTYPE(*LCL) REPLACE(*YES)  TEXT(‘Messages from system A’ )
Transmit Queue	DEFINE QLOCAL(SYSTEMB) REPLACE + USAGE(xmitq) + DESCR(‘Xmit Queue’ )	CRTMQMQ QNAME(B.TO.A) QTYPE(*LCL) REPLACE(*YES) + TEXT(¢ Xmit queue¢ )   USAGE(*TMQ)
Channel A to B	DEFINE CHANNEL(A.TO.B) + CHLTYPE(sdr) REPLACE + TRPTYPE(tcp) CONNAME(9.24.104.116) + XMITQ(SYSTEMB) + DESCR(  ‘Sender channel from A to B’ )	CHLNAME(A.TO.B) CHLTYPE(*RCVR) REPLACE(*YES) TRPTYPE(*TCP) TEXT(‘Receiver channel from A to B’)
Queue 2	DEFINE QLOCAL(‘ Queue2’)   REPLACE + DESCR(‘ Messages from system B’) DEFINE QREMOTE(‘Queue2’ )   REPLACE +	CRTMQMQ QNAME(‘Queue2’ )   QTYPE(*RMT) REPLACE(*YES) + TEXT(‘ Queue 2 in system A’)  + RMTQNAME( ‘Queue2’)   RMTMQMNAME(‘ SystemA’)  + TMQNAME(B.TO.A)
Channel B to A	DEFINE CHANNEL(B.TO.A) + CHLTYPE(rcvr) REPLACE + TRPTYPE(tcp) + DESCR(‘Receiver channel from B to A’)	CRTMQMCHL CHLNAME(B.TO.A) CHLTYPE(*SDR) REPLACE(*YES) + TRPTYPE(*TCP) TEXT(‘Sender channel from B + to A’)   CONNAME(WTR5317) TMQNAME(B.TO.A)

Start Communication Manually

1. Create objects either interactively using  menu CMDMQM or in batch by the OS/400 command STRMQMMQSC (This is done only once to create the objects).
2. Start the listeners and channels. Only the sender channels need to be started manually in both the systems. The receiver channels are started automatically. 
3. The AS/400 listener is started with the command STRMQMLSR.
4. Then start the applications in both the systems.

The commands used to create and start the objects in both the systems are,

	System A	System B (AS/400)
1.	strmqm  SYSTEMA runmqsc <system.tst> a.a	STRMQM STRMQMMQSC  SRCMBR(SYSTEMB)  + SRCFILE(MQLIB/QMQSC)
2.	strmqm SYSTEMA start runmqlsr -t tcp runmqsc start channel(A.TO.B) Ctrl+C	STRMQM STRMQMLSR STRMQMCHL CHLNAME(A.TO.B)

Start Channels Automatically

1. Channel Initiator is used to start Channels Automatically.

STRMQMCHLI QNAME(SYSTEM.CHANNEL.INITQ)

2. The Transmit Queue definition to start the channels automatically is

CRTMQMQ QNAME(B.TO.A) QTYPE(*LCL) +

REPLACE(*YES) TEXT(‘Xmit queue’ ) +

TRGENB(*YES) TRGTYPE(*ALL) +

INITQNAME(SYSTEM.CHANNEL.INITQ)

3. The queue manager can trigger the process that starts

     the channel program in three ways:

· When the first message is put into the transmission queue (*FIRST)

· Every time a message is put into the xmit queue (*ALL)

· When the queue contains a specified number of messages (*DEPTH)

Start Applications Automatically

When a message arrives from another system, the application program that has to process the message can be started manually from the command line or as part of the startup sequence.

Another way is to let the queue manager start it when it is needed, that is, using the MQSeries triggering mechanism.

1. The listener program monitors the port assigned to MQSeries, 1414, and invokes the message channel agent (MCA) when a message arrives.
2. The message is moved into the appropriate local queue (specified in the message header) and a trigger message is put into an initiation queue.
3. The Trigger monitor monitors the initiation Queue and starts the program. The program name is specified in the process.
4. The program then gets the message off the queue.

Client Server Connection

To form a Client Server connection, the communication protocol and the addresses of the systems to be connected are to be known.

Definitions in Server

1. Define the queues that the application needs and a channel of the type

      server connection. 

2. Define a queue for the application to put messages in .

DEFINE QLOCAL(¢ JAVAQ1¢ )   REPLACE +

DESCR(‘ Queue for Java Application’ )

3. An MQI channel of the type server connection.

DEFINE CHANNEL(‘ JAVACH1 ’)   CHLTYPE(SVRCONN) REPLACE +

TRPTYPE(TCP) MCAUSER(‘  ’ )

Definitions in Client 

1. Define an environment variable for the MQSeries client that defines the connection on the client side. Set the variable with the following command or place the command in the CONFIG.SYS.

          Set MQSERVER=JAVACH1/TCP/9.24.104.206(1414)

1. MQSERVER is the name of the environment variable.

2. JAVACH1 is the name of the channel to be used for communication between client and server. The channel must be defined in the server.

3. TCP denotes that TCP/IP is to be used to connect to the machine with the address following the parameter.

4.  (1414) is the default port number for MQSeries.

 

Start Client Server Connection

1. Before the application in the client is started , the Listener program in the server that listens to the communication link between client and server is to be started.

start runmqlsr /t tcp /m JAVAMQM /p 1414

2. The server can then be ready to process MQI calls from the application running in the client.

Working of Client/Server Connection

1. All clients use the queue manager in the server machine as there are MQI channels of type Server Connection between the client and Server.

2.  When a client puts a message on a queue it is to be read and processed by some program. This program can be started when the server starts or the queue manager can start it when needed by using the MQSeries triggering mechanism.

3. The below  Figure shows two clients connected to a server. Both clients request services from the same program (Appl S1).

4. Since that application runs in the same system as the queue manager, we have only local queues. Some queues are specifically for a particular client, for example, QA1 is the reply queue for client A and QA2 is the reply queue for client B.

5. Other queues are used by both clients and server. For example, QS1 is used as output queue for both clients and as input queue for the server program.

Requests sent by Client

1. The client starts a program that puts a message on a queue. For this function five MQSeries API calls are executed:

• MQCONN connects to the queue manager in the server.
• MQOPEN opens the message queue for output (QS1).
• MQPUT puts the message on the queue.
• MQCLOSE closes the queue (QS1).
• MQDISC disconnects from the queue manager (JAVAMQM).
1. The program can put many messages in the queue before it closes it and disconnects. Closing the queue and disconnecting from the queue manager could be done when the application ends.

3. The MQSeries client code that runs in the client machine processes the API calls and routes them to the machine defined in the environment variable.

Server Receiving Requests

1. In the server machine, the following queue manager objects are needed:
• A channel, JAVACH1, of the type server connection.
• A local queue, QS1, into which the clients put their messages.
• An initiation queue into which the queue manager puts a trigger message when a request for queue QS1 arrives.  
• Initiation queue (We use the default initiation Queue)
• A process definition,  process.S1, that contains the name of the program to be started when the trigger event occurs (S1).
• One or more queues in which the program puts the reply messages (QA1 and QB1).

DEFINE CHANNEL(‘ JAVACH1’ )   CHLTYPE(SVRCONN) REPLACE +

TRPTYPE(TCP) MCAUSER(‘   ‘ )

DEFINE QLOCAL(‘ QS1’ )   REPLACE +

DESCR(‘ Queue for Server Application’)  +

TRIGTYPE(EVERY) +

TRIGGER INITQ (system.default.initiation.queue) +

PROCESS (process.S1)

DEFINE PROCESS(process.S1) REPLACE +

DESCR(‘ Process to start server program’)  +

APPLTYPE (OS2) +

APPLICID(‘ c:\mqtest\s1.exe’ )

DEFINE QLOCAL(‘ QA1’ )   REPLACE +

DESCR(‘ Reply queue for client A’ )

DEFINE QLOCAL(‘ QB1’ )   REPLACE +

DESCR(‘ Reply queue for client B’ )

2. In the server machine, two programs have to be started:
• The listener with the command runmqlsr /t tcp
•  The trigger monitor with the command start runmqtrm

3. The listener listens for messages on the channel and puts them on the queue QS1.

4. Since QS1 is triggered, the MQM puts a trigger message on the trigger queue each time a message is put on QS1. When a message is placed on the trigger queue, the trigger monitor starts the program defined in the process.

5. The server program S1 connects to the queue manager, opens the queue QS1 and issues an MQGET to read the message.

Syncpoint and AS/400 Commit Control

1. The data is as an MQ queue as it is in an AS/400 database file. This is done by  synchronizing the transactions, for example, if records from a file are read and put in a queue, then data from a single transaction should be:
• In the queue and not in the file, if the transaction is completed.
• In the file and not in the queue, if the transaction has failed.

2. The standard AS/400 operations for commit control also work for MQSeries for AS/400. Therefore, the MQ operations MQBEGIN, MQCMIT, and MQBACK are not needed.