SaveISISReflectometryORSO v1#

Summary#

Saves ISIS processed reflectometry workspaces into either the ASCII or Nexus implementation of the ORSO data standard.

Properties#

Name

Direction

Type

Default

Description

WorkspaceList

Input

str list

Mandatory

A list of workspace names containing the reduced reflectivity data to be saved.

MetadataSource

Input

string

History

Where the metadata in the file should be taken from. History: Takes from the workspace history (Relies on ReflectometryReductionOne). HistoryWherePossible: Checks workspace history, then prompts for manual input for anything not gathered.Manual: Requires all metadata to be given as properties to this algorithm. Allowed values: [‘History’, ‘HistoryWherePossible’, ‘Manual’]

IgnoredProperties

Input

str list

Properties that are not mandatory when using the ‘HistoryWherePossible’ metadata source.

DatasetSpecificMetadata

Input

string

A JSON string for assigning metadata to the datasets that will be produced by the given WorkspaceList. Please see the algorithm documentation for details on how this JSON string should be structured.

QConversionMethod

Input

string

RefRoi

The method used for converting from Q Space to Wavelength. Allowed values: [‘RefRoi’, ‘ConvertUnits’]

ReductionTimestamp

Input

string

__manual_metadata_not_set__

When the reduction took place (ISO8601 Standard).

FirstTransmissionFileList

Input

str list

List of files used in the first transmission run.

SecondTransmissionFileList

Input

str list

List of files used in the second transmission run.

FloodCorrectionSource

Input

str list

Entry for the flood correction. Must be in the form ‘filename, comment’.

CalibrationFile

Input

string

__manual_metadata_not_set__

Calibration file used.

WriteResolution

Input

boolean

True

Whether to compute resolution values and write them as the fourth data column.

IncludeAdditionalColumns

Input

boolean

False

Whether to include the four additional columns lambda, dlambda, theta and dtheta for unstitched datasets.

Filename

Input

string

Mandatory

File path to save the ORSO file to. Must end with a supported ORSO file extension. Use .ort to save into the ASCII format or .orb to save into the Nexus format. Allowed extensions: [‘.ort’, ‘.orb’]

ModelDescription

Input

string

The model description of the sample.

ValidateModel

Input

boolean

False

Whether or not to validate the model provided to ModelDescription.

Description#

This algorithm saves the data and relevant metadata from a reduced ISIS Reflectometry workspace into either the ASCII (.ort) or Nexus (.orb) implementation of the ORSO data standard [1]. The orsopy library [2] is used to format and write out the file. Currently this algorithm is unable to collect all of the mandatory information required by the ORSO standard so it produces a file that is not fully ORSO compliant. A comment is included at the top of the file to reflect this. This algorithm is only suitable for use with Reflectometry data collected at the ISIS Neutron and Muon facility.

The WorkspaceList passed to the algorithm should be a list of one or more reduced Reflectometry workspaces, or workspace groups, in units of momentum transfer (\(Q\)). For each individual workspace (When in History mode for the MetadataSource property), the algorithm attempts to find metadata for the file header and the resolution (\(dQ/Q\)) from the instrument, workspace history and sample logs associated with the workspace. As a result, this algorithm may produce a file that is missing information if it is passed a workspace that wasn’t reduced via the ISIS Reflectometry Interface. See below for further information about where the metadata and resolution are searched for in the workspace.

If more than one workspace name is passed in then the algorithm will save all the data and metadata to a single ORSO file as a series of datasets. Dataset names must be unique, so an error will be thrown if duplicate dataset names are generated from the workspaces in the list. See below for more information about how these are generated.

When passed a workspace group, the algorithm will save each workspace in the group as an individual dataset within the same file, as described above.

The Filename property is used to provide the save location and name for the file. The save path must end with a valid ORSO file extension - use .ort to save into the ORSO ASCII format or .orb to save into the ORSO Nexus format.

Dataset names#

A dataset name is generated automatically for each individual workspace in the input list or workspace group. This is done as follows:

  • If there is a call to Stitch1DMany v1 in the workspace history then the dataset is given the name “Stitched”.

  • If it is not a stitched dataset then, if available in the workspace reduction history, the value of theta that was used for conversion to Q is given as the dataset name.

  • If a dataset name cannot be generated from either of the above, then the workspace name is used as the dataset name.

For a workspace that is a member of a workspace group, if the generated dataset name is either “Stitched” or the theta value, we include the following additional information in the name to make it unique:

  • If the data is polarized, then we include the spin state using the ORSO notation for polarization.

  • If the data is not polarized, then we include the individual workspace name.

Depending on the combination of workspaces and workspace groups passed in, it is possible that duplicate dataset names will be generated. If this happens then the algorithm will give an error and fail to save out a file.

Metadata Sources#

In its default state, the algorithm will attempt to get as many of the metadata as it can from the workspace history, which requires that the workspace has been processed using the same workflow used in the ISIS Reflectometry Interface. This is not compatible with all reduction workflows, however, and so this algorithm also provides methods for fine-tuning where the saving process sources any metadata.

MetadataSource

Collection Process

History

The input workspace’s history and sample logs will be searched for each of the metadata items. Any that cannot be found will be omitted from the final saved ORSO file.

HistoryWherePossible

As above, but any missing metadata will be marked as an error on the relevant algorithm property during validation. If any marked properties are not expected to be found or are not required, a list of those properties names should be given as inputs to IgnoredProperties.

Manual

Only the values entered for the relevant properties will be included in the output file. A list of the metadata and their relevant properties is included below.

Data values#

The saved ORSO file contains at least three columns of data: the normal wavevector transfer (\(Qz\)), the reflectivity (\(R\)) and the error of the reflectivity. The data is converted to point data using algorithm ConvertToPointData v1 before being saved to file.

If parameter WriteResolution is set to True then the algorithm will also attempt to include a fourth column that calculates the resolution of the normal wavevector transfer as: \(resolution * Qz\). The resolution (\(dQ/Q\)) is looked up as follows:

  • In History or the first step of HistoryWherePossible mode:

    • Find the last occurrence of Stitch1DMany v1 in the workspace history. If this can be found, then the absolute value of the stitch Params parameter is used for the resolution.

    • Otherwise, find the last occurrence of ReflectometryReductionOneAuto v3. This algorithm makes a call to Rebin v1 and the absolute value of the middle rebin Params parameter is used as the resolution.

  • In Manual or the manual step of HistoryWherePossible mode:

    • The value is taken from the Resolution property.

If a resolution value cannot be found from the workspace history or from the property then the file is saved without this column included.

If parameter IncludeAdditionalColumns is set to True then, for non-stitched datasets, the algorithm will attempt to output the additional columns as follows. This setting is independent of WriteResolution: the Qz resolution column is only written when WriteResolution is set to True.

  • lambda - the wavelength values.

    • If the original conversion to Q was performed using RefRoi v1 then the Qz column values are converted back to wavelength using: \(\lambda=\frac{4\pi}{Q}sin(\theta)\). In Manual mode, or the manual step of HistoryWherePossible mode, this angle is taken from the angle values in DatasetSpecificMetadata. If no angle can be found then the additional columns are excluded and a warning is logged.

    • If the original conversion was performed using ConvertUnits v1 then this algorithm is used to convert back to wavelength.

    • Note: The method used is either determined from the workspace history (in History or HistoryWherePossible modes) or from the QConversionMethod property (in Manual or HistoryWherePossible mode. When using the HistoryWherePossible mode, the value in the workspace history will override the setting chosen in the property.)

  • error of lambda - currently assumed to be 0.

  • incident theta - the value of theta used for the final conversion to \(Q\).

  • error of incident theta - calculated as \(resolution * \theta\).

If it is not possible to calculate the values for the additional columns then a warning is logged and they are excluded from the file. For stitched datasets, additional columns are not included and a warning is logged if IncludeAdditionalColumns is set to True.

Header Metadata#

Some of the metadata for the ORSO file header is retrieved directly from the input workspace, as detailed below. For values retrieved from the workspace history, in History mode, if any information cannot be extracted from the history then the file is saved without this metadata included.

Header value

Workspace location

Manual Metadata Property

instrument

The name of the instrument associated with the workspace.

N/A

start_date

The value of the run_start sample log.

N/A

proposalID

The value of either the rb_proposal or experiment_identifier sample log.

N/A

sample name

The workspace title (same as the value of the run_title sample log).

N/A

reduction timestamp

The execution time of the last occurrence of ReflectometryReductionOneAuto v3 in the workspace history.

ReductionTimestamp

measurement additional_files

The individual file names for all of the run numbers passed to parameters FirstTransmissionRunList and SecondTransmissionRunList from all calls to ReflectometryISISLoadAndProcess v1 in the workspace history. Also the flood correction workspace or file name and the calibration file name from ReflectometryISISLoadAndProcess v1 in the workspace history.
FirstTransmissionFileList
SecondTransmissionFileList
FloodCorrectionSource
CalibrationFile

Manual Metadata Blank Values#

In HistoryWherePossible mode, manual metadata properties only override values found in the workspace history if the property is explicitly set. Leaving a property at its default value means that the history value is used where possible. An explicitly set blank value can be used to clear a value found in the history.

Property

Default behaviour

Explicit blank override

DatasetSpecificMetadata

No dataset-specific values are overridden.

{} or an omitted dataset leaves history values unchanged. Present JSON keys override history values; omitted JSON keys do not clear history values.

QConversionMethod

RefRoi is used if no method is found in the history.

No blank override is available. The value must be either RefRoi or ConvertUnits.

ReductionTimestamp

The history timestamp is used where possible.

"" clears the history timestamp.

FirstTransmissionFileList

The first transmission files from history are used where possible.

[""] clears the first transmission file list.

SecondTransmissionFileList

The second transmission files from history are used where possible.

[""] clears the second transmission file list.

FloodCorrectionSource

The flood correction from history is used where possible.

[""] clears the flood correction.

CalibrationFile

The calibration file from history is used where possible.

"" clears the calibration file.

Dataset-Specific Metadata#

Some metadata is specific to each dataset in the output file. There are one of these datasets per workspace given in the WorkspaceList. A WorkspaceGroup has a dataset in the output file for each of its child workspaces. In History mode, these are aquired from the workspace history and the sample logs if available. In Manual mode, these metadata can be provided via a JSON string to the DatasetSpecificMetadata property.

Metadata Fields#

Key

Description

Example Value

ORSO Key

<WORKSPACE_NAME>

The primary key for the dataset’s metadata is the workspace associated with the output dataset.

{ <REST OF JSON> }

N/A

"group-members"

If the primary key is a workspace group, then this JSON object should be used to define the "dataset-name" and "polarization" for the child workspaces.

See template.

N/A

"dataset-name"

The name the dataset should have in the output file.

"13460 0.5"

data_set

"is-stitched"

If the dataset is a stitched set. Affects sorting and if additional columns are included.

true

N/A

"polarization"

The spin states used in the experiment for this dataset. See the orsopy documentation for details on the expected formats.
unpolarized
mp, etc.

polarization

"resolution"

The resolution value used to calculate the fourth data column (if WriteResolution is true). See above for details.

1.82

N/A

"data-files"

A list of JSON objects defining the data (/angle/run) files associated with the dataset.

[{ "file-name : "x", "angle" : 0.0 }]

data_files

"file-name"

The name of the data file.

"INTER00013460"

file

"angle"

The angle of the data file. This is also used to calculate lambda when using RefRoi with IncludeAdditionalColumns.

0.5

comment

"reduction-call"

The script used to perform the reduction.

"ReflectometryI...""

call

Example#

{
  "IvsQ_13460" : {
    "dataset-name" : "IvsQ_13460",
    "is-stitched" : false,
    "polarization" : "unpolarized",
    "resolution" : 0.5,
    "data-files" : [
      {
        "file-name" : "INTER00013460",
        "angle" : 0.5
      }
    ],
    "reduction-call" : "ReflectometryISISLoadAndProcess(InputRunL..."
  },
  "IvsQ_47041_47042" : {
    "group-members" : {
      "IvsQ_47041_47042_1 (Workspace 2D)" : {
        "dataset-name" : "47041_47042 Plus Plus Stitched",
        "polarization" : "pp"
      },
      "IvsQ_47041_47042_2 (Workspace 2D)" : {
        "dataset-name" : "47041_47042 Plus Minus Stitched",
        "polarization" : "pm"
      }
    },
    "is-stitched" : true,
    "data-files" : [
      {
        "file-name" : "POLREF00047041",
        "angle" : 0.4
      },
      {
        "file-name" : "POLREF00047042",
        "angle" : 1.0
      }
    ],
    "reduction-call" : "ReflectometryISISLoadAndProcess(InputRunL..."
  }
}

Templates#

Matrix Workspace#

{
  "<WORKSPACE 1 NAME>" : {
    "dataset-name" : "<DATASET NAME (string)>",
    "is-stitched" : <true/false (bool)>,
    "polarization" : "<POLARIZATION MARKER (string)>",
    "resolution" : <RESOLUTION (float)>,
    "data-files" : [
      {
        "file-name" : "<RUN FILE NAME (string)>",
        "angle" : <RUN FILE ANGLE (float)>
      }
    ],
    "reduction-call" : "<REDUCTION SCRIPT (string)>"
  }
}

Workspace Group#

{
  "<WORKSPACE GROUP NAME>" : {
    "group-members" : {
      "<CHILD WORKSPACE 1 NAME>" : {
        "dataset-name" : "<DATASET_NAME (string)>",
        "polarization" : "<POLARIZATION (string)>",
      },
      "<CHILD WORKSPACE 2 NAME>" : {
        "dataset-name" : "<DATASET_NAME (string)>",
        "polarization" : "<POLARIZATION (string)>",
      }
    },
    "is-stitched" : <true/false (bool)>,
    "data-files" : [
      {
        "file-name" : "<RUN FILE NAME (string)>",
        "angle" : <RUN FILE ANGLE (float)>
      },
      {
        "file-name" : "<RUN FILE NAME (string)>",
        "angle" : <RUN FILE ANGLE (float)>
      }
    ],
    "reduction-call" : "<REDUCTION SCRIPT (string)>"
}

Usage#

Example - Save a workspace in ISIS Reflectometry ORSO ASCII format

# import the os path libraries for directory functions
import os

ws = CreateSampleWorkspace(XUnit="MomentumTransfer", NumBanks=1, BankPixelWidth=1)

# Create an absolute path by joining the proposed filename to a directory
# os.path.expanduser("~") used in this case returns the home directory of the current user
# Specify the .ort extension to save to the ORSO ASCII format
file = os.path.join(os.path.expanduser("~"), "ws.ort")

# Add Sample Log entries
AddSampleLog(Workspace=ws, LogName='rb_proposal', LogText='1234', LogType='Number')

# Save the ORSO file
SaveISISReflectometryORSO(WorkspaceList=ws, Filename=file, WriteResolution=False)

# Open the file and read the first line
if os.path.exists(file):
  with open((file), 'r') as myFile:
    print(myFile.readline())

# # ORSO reflectivity data file | ... standard | YAML encoding | https://www.reflectometry.org/

References#

Categories: AlgorithmIndex | Reflectometry\ISIS

Source#

Python: SaveISISReflectometryORSO.py