Server Management ***************** *NXRefine* implements a data reduction workflow, which can be run as a series of line commands in the terminal. However, since some of the processes can take a long time to complete (from a few minutes to an hour, depending on the process and system being used), it is possible to queue these operations using the *NXRefine*'s queue manager, to be run locally using multiple cores or distributed to other nodes. The *NXRefine* queue manager can be configured to submit jobs to another job queue manager if one is available. Server Directory ================ Here is the structure of the ``nxserver`` directory:: nxserver ├── nxserver.log ├── cpu1.log ├── cpu2.log ├── cpu3.log ├── settings.ini ├── nxsetup.sh ├── nxcommand.sh └── task_list ├── info └── q00000 └── locks ├── ... └── ... **nxserver.log** This is a log file that records jobs submitted to the server queue. **cpu1.log**, **cpu2.log**, ... These are log files that contain the output of jobs running on the server. The number depends on the number of simultaneous jobs that are allowed on the server, which is defined by the settings file. **settings.ini** A file containing default settings used by the *NXRefine* package, including server parameters, instrumental parameters, and parameters used in the data reduction workflow. When a new experiment is set up, a copy of these parameters is stored in the experiment directory (to be described later), so that they can be customized if necessary. These settings are described below. **nxsetup.sh** A shell script that could be used to initialize paths to the server directory or environment variables used by *NeXpy*. This could be run within a user's ``~/.bashrc`` file, or by other shell scripts used to launch *NXRefine* workflow jobs (see below). Here is an example of what this file could contain.:: export NX_LOCKDIRECTORY=/path/to/parent/nxserver/locks export NX_LOCK=10 export NX_SERVER=/path/to/parent/nxserver Other commands, *e.g.*, to initialize a particular conda environment, could be also be added to this file. **nxcommand.sh** A shell script that is used if jobs need to be wrapped before submission to the job queue, *e.g.*, using ``qsub``. Here is an example, in which ``nxsetup.sh`` is run in order to initialize *NXRefine*.:: echo `date` "USER ${USER} JOB_ID ${JOB_ID}" source /path/to/parent/nxserver/nxsetup.sh **task_list** A directory that contains files that implement a file-based FIFO queuing system for server jobs. **locks** A directory that contains files that implement the `nexusformat `_ file-locking system. Locked files can be viewed, and removed if they are stale, using the "Show File Locks" dialog in the *NeXpy* "File" menu. .. note:: The log files can be viewed using the "Manage Server" dialog and the settings file can be modified using the "Edit Settings" dialog, both of which are located in the "Server" menu in *NeXpy*. .. figure:: /images/server_settings.png :align: right :width: 90% :figwidth: 50% .. _default_settings: Default Settings ================ The file, ``settings.ini`` in the server directory contains the default settings for the server, the beamline, and the workflow. These values can be changed, either by opening the "Edit Settings" dialog in the *NeXpy* "Server" menu or at the command line using ``nxsettings -i``, which lists all the settings one by one, allowing their values to be changed. Hitting the [Return] key keeps the current value. The figure shows an example of the first two sections of ``settings.ini``. The parameters in the first section are described here. The other sections contain information concerning the location of the data and default values of the data reduction parameters. They will be described later. Server Settings =============== The server settings are used by the workflow server, which is described in a later section. They define the server configuration, such as the number of simultaneous jobs that may be run, the command required to add them to the system's standard job queues, and whether they need to be wrapped in a shell script. :type: The server type can either be ``multicore`` or ``multinode``. The only difference is that multinode servers have a list of defined nodes, to which jobs may be submitted, so their names will also be stored in the settings file. If jobs are submitted to a job server, without needing to specify the node, or if all the jobs are performed on the local machine, then the server type should be ``multicore``. :cores: This sets the number of jobs that can be run simultaneously by the server. Once reaching the limit, new jobs will only start as old ones are finished. :concurrent: This determines whether parallelized processes should be used in the workflow. These speed up the computation, but can be disabled if they cause issues with the server. Note that this refers to whether multiple processes can be run simultaneously, *e.g.*, in peaks searches, not whether multiple jobs can be submitted to the server. Valid values are ``True`` or ``False``. :run_command: This is a string that is prepended to any jobs that are submitted to the server. It can contain a set of switches in addition to the job submission command itself. :template: In some systems, it is necessary to wrap the command that is submitted to the server in a shell script. This is the name of the script, which should be stored in the ``nxserver`` directory. It should contain the string ````, which is replaced by the job command. :cctw: This is the path to the CCTW executable used to transform data from instrumental coordinates to reciprocal space. Server Menu =========== The *NXRefine* plugin to *NeXpy* installs a top-level menu labelled "Server", which is used to launch and monitor data reduction operations performed as part of the workflow. Manage Workflows ---------------- This dialog shows the workflow status for all the scans stored in a particular sample directory. When the dialog is launched, click on "Choose Sample Directory" to launch the system file browser in order to select a directory containing a set of NeXus scan files. This opens the "Manage Workflows" dialog, in which the status of every component of the *NXRefine* workflow is listed for all the scan files in the sample directory. Manage Server ------------- This dialog allows the status of the server to be monitored.