commhub

Commhub Daemon

The commhub daemon is the task controller and communications hub for all the other daemons for MAE (Multi-Application Ecosystem). No other daemons can run without commhub running.

Communication to commhub is through a TCP connection or Unix socket (/usr/mae/conf/s.commhub).

Commhub supports three functional communication daemons - dbbroker, msgbroker, rpcbroker. The Db*, Msg* and Rpc* commhub commands are not processed by commhub, but handed to the respective functional daemon to process. dbbroker manages data storage. msgbroker handles messaging amongst daemons and applications. And rpcbroker handles remote procedure calls where an app calls a service and waits for the reply. Messages are for asynchronous (one-way communication; remote procedure calls are for synchronous (two-way) communiocation.

Use the tstatus, tstart, and tstop utilities to control tasks from the command line. Use tviewlog to view logged activity.

Command Line Arguments

Argument	Description
-nofork	Do not fork() commhub upon starting.
-start	Start all auto-start daemons upon starting.
-hub	Declare this commhub as root commhub. This is the default. Compare to -spoke.
-spoke	Declare this commhub as a spoke commhub that must connect to the root commhub to be functional. If the connection to the root is broken unexpectedly, a suspend is sent to all local tasks while commhub tries to reconnect to the root hub.
-h [hostname]	Declare root commhub's hostname/address. If not specified, then the environment variable commhost will be used. Use with -spoke command line option.
-p port	Specify the TCP port to listen on. The default is 4847.
-d dir	Specify the commhub configuration directory, e.g. /usr/mae/conf. This should be the first command line argument.
-s	Commhub will listen for connections for any IP host. This should only be used in a secure environment.
-l	Commhub will listen for connections only from localhost (127.0.0.1).
-g	Start commhub into debug mode. This implies the -nofork option.

Programming API to CommHub

To communicate with CommHub to manage and control its tasks, use the MAETask class. Also use the MAETask class to query and set MAE system parameters (global variables for apps to use for configuration).

To send messages, see the MsgBroker documentation.

To send service requests with data and then receive response data back, see the RpcBroker documentation.

To query the database, see the DbBroker documentation.

Database Tables

CommHub does not use any database tables or other middleware. It uses CSV files as database tables for tasks.

Configuring Commhub

Commhub reads these files upon startup:

Filename	Description
/usr/mae/conf/ mgmttaskdb.csv	A database of tasks allowed to run and their permissions. Required.
/usr/mae/conf/taskdb.csv	A database of the current state of commhub's tasks.
/usr/mae/conf/ commhub.ini	Global variables for the MAE environment that daemons and applications can query.
/usr/mae/log/commhub.log	The log file for commhub major events and all daemon and application logged messages.
/usr/mae/log/commhubdbg.log	Commhub debug log (when -g flag used)

Commhub.ini

This file contains initial values for commhub and the applications that it supports. The file format is INI format:

A comment line is ignored; it begins with a #
A section contains a series of key/value pairs. A section begins with a keyword inside square brackets, e.g. [appname]. The keyword is an application's name.
A key/value pair is separated by equals sign, e.g. =. The keyword may contain letters, numbers, periods, and dashes. The value is all text following the first equals sign on the line.
The special keyword spokes defines a series of spokes that are allowed to connect to commhub. Each key specified is the spoke's hostname; each value is the spoke user allowed to connect (typically mae).
The special keyword commhub is reserved.
Any key/value pairs before the first second is considered the global variable section. All key/value pairs in the global section are defined and queryable by all applications.

mgmttaskdb.csv

This file is located in /usr/mae/conf/mgmttaskdb.csv; it contains the list of daemons and tasks that this commhub and its spokes collectively manage. The fields are:

Field Name

Description

Unique record id, starting at 1.

Name

The name for the daemon or task. No dots or spaces allowed in name.

Restart

When commhub detects an unexpected exit by the daemon/task, this value determines if it should be automatically restarted. The value must be T for true or F for false.

Enabled

This flag determines whether commhub will manage the daemon/task. Unmanaged tasks are ignored. The value must be T for true or F for false.

MultiExec

This flag determines whether a daemon/task can be run with instances. The value must be T for true or F for false.

DbPermit

This flag determines whether the daemon or task is permitted to access the database / datastore. The value must be T for true or F for false.

MsgPermit

This flag determines whether the daemon or task is permitted to send/receive messages. The value must be T for true or F for false.

Command

This is the command to execute to start the daemon/task. It is wise to include the path before the program name. Command line arguments may be provided, but commhub may insert "-instance n" as the first parameter when launching instances.

Integration

The level of integration supported by the daemon. Integration is required for commhub to understand a daemon's or task's status as well as controlling the daemon or tasks.

Level	Description
1	Fully integrated. The daemon will connect with commhub to send init/ready/done notifications and the daemon will respond to commhub commands of suspend/resume/stop/debug/etc.
2	Semi-integrated. The daemon will not connect directly to commhub, but it will keep commhub updated (such as by using the tctrl command line utility) to report init/ready/done status. It cannot/will not respond to commhub commands.
3	Not integrated. This daemon or task is not and cannot be controlled or updated by commhub. It will run and complete on its own.

DepList

A list of daemons or tasks by name that must be running before this daemon or task can run. In other words, it depends upon this list of other daemons / tasks.

ClusterName

This specifies the commhub spoke name where the daemon or task should run by default. It may be blank or root to run on the root commhub.

ExitTimeout

Number of seconds to allow when commhub has instructed the daemon to stop before commhub escalates to sending a TERM signal to the process and then wait again before sending a KILL signal to it. For example: 1.

WatchdogTimeout

Number of seconds to allow since the last response from a daemon or task before commhub kills it. This is effective for terminating processes that are unresponsive. For example: 60.

ProxyHost

Not implemented since ClusterName is typically used for locating where daemons and tasks are running, if not on the root commhub.

Schedule

Not implemented. Specifies when a daemon or task should be started if not running. Value specified in military time, e.g. 7:00 or Mon 8:00. If a range is provided, then the daemon will be started if it is running at the beginning of the time range and it will be stopped if it is still running at the end of the range, e.g. 8:00-18:00.

Commhub Commands

Commhub accepts connections from MAE daemons on its TCP port or Unix socket. Commhub accepts the following commands on input.

A frompath is used to direct messages between multiple commhub instances. There is a central, root commhub instance with spokes that connect to it. The path lists the commhub nodes to traverse to get to the destination, e.g. root!nice!22 first sends the message to the root commhub, then to the nice spoke, then it local app id 22 there.

Command	Params	Description
DbOpen	[frompath appname]	Initiate database session from appname. A session contains app-specific context, such as current table. Routed to dbbroker. See dbbroker for actual request. The frompath serves as the session id. Note that the frompath and appname are used for communication between commhubs and to dbbroker itself.
DbReq	[frompath appname] <DBRequest ... >	Database request/query. Routed to dbbroker. See dbbroker for actual request. The frompath directs DbResp along that path. A DbResp command is issued to the frompath to send the response back. Note that the frompath and appname are used for communication between commhubs and to dbbroker itself.
DbResp	frompath + <DbResponse ... > frompath -error-message	DEPRECATED. Response from database request from dbbroker. See dbbroker for actual response. The frompath points to the app awaiting the response. Note that the frompath and appname are used for communication between commhubs and from dbbroker itself. Deprecated because dbbroker now uses relay/DbResp.
DbClose	[frompath appname]	Finalize database session from appname. Routed to dbbroker. See dbbroker for actual request. The frompath serves as the session id. Note that the frompath and appname are used for communication between commhubs and to dbbroker itself.
MsgListen	frompath appname src	A daemon or appication is opening a communication channel (src); it is telling msgbroker its id (frompath) and requesting messages to its channel be sent to it. Commhub sends this to msgbroker for processing and setup the channel.
MsgSend	topath appname src dest message	DEPRECATED. msgbroker uses this commhub command to send the message to the intended daemon or application at topath that listens to channel (dest). Deprecated because dbbroker now uses relay/MsgSend.
MsgRoute	frompath appname dest message	An daemon or application (frompath) is sending a message to another daemon/application's channel (dest) from its channel (src). Commhub sends this to msgbroker for processing.
MsgClose	frompath appname src	A daemon or application (frompath) is closing its communication channel (src). Commhub sends this to msgbroker for processing to close down the channel.
RpcService	frompath appname sevice timeout	Message from an app to register a new RPC (Remote Procedure Call) service with rpcbroker. A service is identified by a keyword without spaces (it may contain dashes or dots).
RpcReq	frompath service appname timeout params	Message from an app to send an RPC request via rpcbroker.
RpcResp	frompath appname service reply	Message from a service to rpcbroker to with reply from RPC service app.
RpcOK	reply	DEPRECATED. Message from rpcbroker with success response from RPC service request. Sent via relay command.
RpcError	reply	DEPRECATED. Message from rpcbroker with failure response from RPC service request. Sent via relay command.
relay	dest message	Internally relay a message between commhub instances to a client app. dest is the sessionid of the app. It may be prepended by cluster name, e.g. root!35 or root!spoke!35.
log	"message"	Record a message in commhub.log. The time and source daemon is also recorded.
get	taskname varname	Get a system parameter, which is stored in memory and /usr/mae/conf/commhub.ini.
set	taskname varname value	Set a system parameter.
start	taskname [frompath]	Start a daemon or application. The taskname must be a configured name for commhub (see mgmttaskdb.csv). It may optionally contain an instance suffix, e.g. ngiapp.13. It may optionally contain a spoke prefix to specify where it should run, e.g. nice.home:ngiapp. The tstart utility uses this. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
stop	taskname [frompath]	Stop a daemon or application. See start for valid tasknames. The tstop utility uses this. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
suspend	taskname [frompath]	Suspend a daemon or application. See start for valid tasknames. The tsuspend utility uses this. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
resume	taskname [frompath]	Resume a daemon or application. See start for valid tasknames. The tresume utility uses this. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
debug	taskname level [frompath]	Switch a daemon or application into or out of debug mode. See start for valid tasknames. Additionally, commhub may be specified. The tdebug utility uses this. When in debug mode, the daemon or application store extra information into a special logfile in /usr/mae/log. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
tap [frompath]	taskname	Tap into database, message, and RPC communication going through commhub. These taped messages may then be replayed by msgplayer to assist with message testing. See start for valid tasknames. The from path directs return code along that path. Return code is +OK upon success or -message upon failure.
list		Provide a list of daemons and applications running.
status	taskname	Provide the status of a daemon or application. See start for valid tasknames.
instances	taskname	Return of list of running instances for the taskname. These instances are in addition to the base instance that status reports. No every task has instances.
user	username	Used to declare what user is connected on the socket; used for permission access. Unix user id expected as parameter.
daemon	username	Used to declare the connection is by a daemon or application. Unix user id expected as parameter.
spoke	username	Used to declare the connection is a spoke/satelite commhub connection. Unix user id expected as parameter.
spokelist	from	Produce a list of connected spokes to this commhub, with the output sent to the optional from commhub. Output is spoke name, space, status. Status is on-line or off-line.
notify-init	taskname appname pid	Message from app to inform commhub that the named task is initializing itself. The process has started, but it's not fully ready for messages yet.
notify-ready	taskname "message" [node]	Message from app to inform commhub that the named task is fully ready. It can start receiving messages and other requests. If node is provided in message from a spoke, then node specifies where its running.
notify-done	taskname "message"	Message from app to inform commhub that the named task is finished running. It has completed successfully with no errors.
notify-failed	taskname "message"	Message from app to inform commhub that the named task is finished running. It has exited with errors.
notify-degraded	taskname "message"	Message from app to inform commhub that the named task is experiencing degraded performance. (Future use.)
notify-resume	taskname "message"	Message from app to inform commhub that the named task has resumed normal operstions after being suspended. (Future use.)
notify-suspend	taskname "message"	Message from app to inform commhub that the named task has suspended operstions; it is idle, waiting to be resumed. (Future use.)
notify-pid	taskname pid	Message from app to inform commhub of its process id.
notify-add	taskname	Meessage from tadd that an application or daemon in mgmttaskdb should be promoted to a startable task. It will be added to the taskdb.
notify-del	taskname	Meessage from tadd that an application or daemon in mgmttaskdb should be demoted from a startable task. It will be removed from the taskdb.
notify-cluster	taskname spokename	Report to commhub that the named task is running on the spoke commhub.
notify-suspendable	taskname	Inform commhub that the named task may be suspended.
stat		Output various connection and data statistics that commhub has accumulated.
quit		Instruct commhub to exit.
createtaskdb		Reset the internal taskdb - clear all knowledge of tasks. This precedes the sending of mgmtrow and taskrow commands. This command can only be executed from a root hub connection.
mgmtrow	csv-line	Add/Update a task management/configuration record. Values are provided in CSV format. (Update is not implemented yet.) Field order: Id,Name,Restart,Enabled,MultiExec,DbPermit,MsgPermit,Command,Integration,DepList,ClusterName,ExitTimeout,WatchdogTimeout,ProxyHost,Schedule. For example: 1,dbbroker,T,T,F,F,F,/usr/mae/bin/dbbroker,1,,,3,60,,
mgmtrow	csv-line	Add/Update a active task record. Values are provided in CSV format. If the task record already exists, only the ClusterName, User, and Type fields are updated. Fields order: Recno,Name,State,Pid,RemoteAddr,ClsuterName,User,Type,StatusMessage. For example: 1,dbbroker,OK,26009,0,root,root,,Ready.

Support Run-Time Commands

Once commhub is up and running, there are command line utilities to communicate with it or control it.

Command	Description
tadd	Register a new daemon/task for commhub.
tdel	Remove a registered daemon/task for commhub.
tctrl	Used by semi-integrated applications to communicate current state to commhub.
tstart	Start a daemon or task; whether on the default node, root node, or a spoke; whether the base instance or a numbered instance.
tstop	Stop a daemon or task, if that task is integrated with commhub.
trestart	Stop and then start a daemon or task.
tstatus	View a summary of all registered daemons and tasks.
tviewlog	View recent log messages for daemons and tasks.
tspokes	View a list of all spokes and their conneciton status to the local or specified commhub.
tstat	View connection statistics for the local commhub.
tpget	Get a commhub parameter (value from commhub.ini file).
tpset	Set a commhub parameter (puts value into commhub.ini file).
tdebug	Toggle the debug state of a daemon or integrated task.
tmesg	Send a message into commhub.
ttap	Tap into all messages going to or coming from a specific daemon or task.