Skip to main content
`Ctrl`+`K`

# ng911ok.lib.misc module

authors:

  Riley Baird (OK), Emma Baker (OK)

created:

  August 02, 2024

Miscellaneous helper functions, classes, constants, etc., that support various parts of the Toolkit.

exception AttributeConversionError(original_value: FeatureAttributeValue, target_field: NG911Field, *args)

  Bases: `ValueError`

exception MixedZeroParityError

  Bases: `Exception`

  Exception for use in the following situations involving inconsistent parity:

  - Given an address range, one bound is zero, but the other is non-zero.

  - Given a parity and an address range, the parity is zero, but one or both range bounds are non-zero.

  - Given a parity and an address range, the parity is not zero, but one or both range bounds are zero.

class LoadDataFrame(fc_path)

  Bases: `object`

  fix_df_fields(fields) → DataFrame

class NGUIDAssignMethod(value)

  Bases: `StrEnum`

  Methods of NGUID assignment.

  _generate_next_value_(start, count, last_values)

    Return the lower-cased version of the member name.

  COPY = 'COPY'

    Method that simply copies existing NGUID attributes to an NGUID field. Intended for field mapping operations.

  LOCAL = 'LOCAL'

    Method in which the local ID and agency ID are provided and used to build a complete NGUID. Requires populated `
    Local_ID` and `
    Agency_ID` fields.

  NGUID = 'NGUID'

    Method in which the local ID is extracted from an existing NGUID, which is then used with a provided agency ID to build a complete NGUID. Requires populated `
    Agency_ID` and NGUID fields. Conversion from v2.2 NGUIDs is supported with this method.

  NULL = 'NULL'

    NGUID attributes are set to null.

  SEQUENTIAL = 'SEQUENTIAL'

    Method in which sequential numbers, starting with 1, are assigned as the local IDs and used with a provided agency ID to build a complete NGUID. Requires a populated `
    Agency_ID` field.

class Parity(value)

  Bases: `StrEnum`

  classmethod from_modulo_result(value: int | None | NAType, use_na: Literal[False] = False) → Parity | None
  classmethod from_modulo_result(value: int | None | NAType, use_na: Literal[True]) → Parity | NAType
  classmethod from_modulo_result_series(series: Series) → Series
  _generate_next_value_(start, count, last_values)

    Return the lower-cased version of the member name.

  BOTH = 'BOTH'
  EVEN = 'EVEN'
  ODD = 'ODD'
  ZERO = 'ZERO'

class _AttrsInstanceWithAllowedFields(*args, **kwargs)

  Bases: `Protocol`

  _abc_impl = <_abc._abc_data object>
  _is_protocol = True

class _AttrsInstanceWithFeatureClassPath(*args, **kwargs)

  Bases: `Protocol`

  _abc_impl = <_abc._abc_data object>
  _is_protocol = True

all_values_selected(parameter: Parameter) → bool

  Returns whether a multivalue `arcpy.Parameter` has all values in its filter list selected.

arc_field_details(field: Field) → str

calculate_parity(data: int) → Literal[Parity.ODD, Parity.EVEN, Parity.ZERO]
calculate_parity(data: Sequence[int], summarize: Literal[False]) → list[Literal[Parity.ODD, Parity.EVEN, Parity.ZERO]]
calculate_parity(data: Collection[int], summarize: Literal[True]) → Parity

check_user_fc_for_fields(fc_path: str | None, parameter_field_table: list[list[str]] | list[str], error_list: list[str], std_idx: int = 0, user_idx: int = 1) → tuple[list[str], list[list[str]]]

convert_attribute_value(value: _T_FAV, target_field: NG911Field) → FeatureAttributeValue

  Given an attribute value from a feature, attempt to convert it to the appropriate type indicated by target_field.

  Parameters:

    - value – Attribute value to convert

    - target_field – Field that value should be valid for

  Returns:

    Equivalent value of appropriate type

  Raises:

    AttributeConversionError – If value is not valid for target_field

cursor_apply(table: Path | str, fields: Sequence[str], function: Callable[[_P], _R], **cursor_kwargs) → list[_R]

  Applies function row-wise to records in a table or feature class using `arcpy.da.SearchCursor` and returns the results as a list.

  Note that each record is unpacked when function is called, so function must take one argument for each field given in fields.

  Parameters:

    - table – Table or feature class to load

    - fields – Fields whose values will be passed to function

    - function – Function to apply to each row

    - cursor_kwargs – Additional arguments to pass to `arcpy.da.SearchCursor`

  Returns:

    List of outputs from calling function on each row

ensure_no_blank_strings(df: DataFrame, error_message_start: str | None = None) → None

  Raises a `ValueError` if any values in df are blank strings or consist only of whitespace that indicates that they should instead be null. All columns in df must have the `pandas.Series.str` accessor available.

get_closest_match_insensitive(word: str, possibilities: Sequence[str], cutoff: float = 0.6) → str | None

  Returns the string from possibilities most similar to word, or `None` if `difflib.get_close_matches` doesn’t return any results given cutoff. Unlike `difflib.get_close_matches` alone, however, the function is case-insensitive.

get_field_info_for_fc_creation(fc_fields) → list[list[str | int | None]]

get_project() → ArcGISProject | None

  If this function is being called from within an ArcGIS Pro session, an `ArcGISProject` instance representing the currently-open project is returned. If this function is not being called from within an ArcGIS Pro session, `None` is returned. The result is cached.

if_licensed(tool: Callable[[_P], _R]) → Callable[[_P], tuple[bool, _R | None]]

  Wraps an `arcpy` geoprocessing function in a try/except block that, suppresses ERROR 000824 (indicating an unlicensed tool) and returns `None` in place of the tool’s expected return value.

  The callable returned by this function returns two values. The first, a `bool`, will be `True` unless ERROR 000824 is encountered; `False` indicates that the tool is not licensed. The second is the result of wrapped geoprocessing function (or `None` if the tool is not licensed).

indent(lines: _T_indent, prefix: Literal['\t'] | int = '\t') → _T_indent

  Given a single multiline string, returns it with each line prefixed with a tab character. Given an iterable of strings, returns a list of the input strings, each prefixed with a tab character.

is_typed_dict(obj: Mapping, typ: type[TypedDict])

  Given a `Mapping` instance and a `TypedDict` subclass, return whether obj is an instance of typ.

  Warning

  This function will raise a `TypeError` if typ declares a dict key whose type is a parameterized generic, e.g., `list[str]`. This limitation does *not* apply to `typing.Required`, `typing.NotRequired`, or `typing.Annotated`, but it does apply to their parameters, e.g., `Required[int]` is valid, but `Required[list[int]]` is not.

  Parameters:

    - obj (`Mapping`) – A `Mapping` instance whose items should be checked against a `TypedDict` definition

    - typ (type[TypedDict]) – Any `TypedDict` subclass

  Returns:

    Whether the items in obj conform to typ

na_eq(val1: Any, val2: Any)

  Compares any two values, testing for equality. Unline the `==` operator, however, this function will return `True` if both val1 and val2 are considered `NA` by `pandas`, such as `pandas.NA` and `numpy.nan`.

  Parameters:

    - val1 (Any) – Any object

    - val2 (Any) – Any object

  Returns:

    Whether val1 and val2 are equal or both `NA`

  Return type:

    bool

na_in_list(val1: Any, in1: Any)

na_in_set(in1: Any, in2: Any)

na_to_none(val)

  Returns `None` if val is any value that `pandas` considers to be NA/null, otherwise returns val. This function can be used as an `attrs` converter.

params_to_dict(targets: Sequence[Parameter]) → dict[str, Parameter]

qquote(strings: Iterable[str]) → list[str]

  Surrounds each string in strings with double quotes.

quote(strings: Iterable[str]) → list[str]

  Surrounds each string in strings with single quotes.

series_from_cursor(table: Path | str, index_field: str, value_field: str, name: str | None = None, **cursor_kwargs) → Series

  Returns a `pandas.Series`, where the index comes from index_field and the value from value_field.

strjoin(strings: Iterable[str], joiner: str, *, q=False, qq=False) → str

unwrap(target: Parameter) → ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None | list[ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None]
unwrap(target: Sequence[Parameter]) → list[ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None | list[ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None]]

  Extracts the value(s) from `arcpy.Parameter` object(s). This function can be called with either a single `arcpy.Parameter` or a sequence thereof as the argument for target.

  If target is a single `arcpy.Parameter`, a single value

  Parameters:

    target (Union[Parameter, Sequence[Parameter]]) – Parameter object or Sequence thereof

  Returns:

    List of usable objects representing the inputs

  Return type:

    Union[GPParameterValue, list[GPParameterValue]]

unwrap_to_dict(targets: Sequence[Parameter]) → dict[str, ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None | list[ArcSDESQLExecute | Extent | Field | FieldInfo | FieldMap | FieldMappings | Index | Point | RandomNumberGenerator | RecordSet | SpatialReference | bool | float | int | str | datetime | None]]

validate_field_exists(instance: _AttrsInstanceWithAllowedFields | _AttrsInstanceWithFeatureClassPath, attribute: attrs.Attribute[_T_Field], value: _T_Field) → None

  A function compatible with the `validator` argument of `attrs.Field` in cases where a field name or `NG911Field` instance is passed as the value of the `attrs.Field`. A `ValueError` is raised if value is not among the allowed/existing fields.

  Parameters:

    - instance – An `attrs` object with either an `__ng911_allowed_fields__` attribute or an `__ng911_feature_class_path__` attribute

    - attribute – The `attrs.Attribute` object

    - value – Value of the attribute

wrap_string(string: str, *wrappers: str) → str

  Surrounds string with wrappers. If one argument is provided for wrappers, it will be placed both before and after string. If two arguments are provided, the first will be prepended and the second will be appended to string.

wrap_strings(strings: Iterable[str], *wrappers: str) → list[str]

  Surrounds each member of strings with wrappers. If one argument is provided for wrappers, it will be placed both before and after each member of strings. If two arguments are provided, the first will be prepended and the second will be appended to each member of strings.

DOMAIN_TYPES: FrozenDict[str, str] = FrozenDict({'CODED': 'CodedValue', 'RANGE': 'Range'})

  Mapping of domain type keywords where keys are those used in domain creation and values are the corresponding keywords found in `arcpy.da.Domain.domainType`.

FIELD_DATA_TYPES: FrozenDict[Literal['SHORT', 'LONG', 'BIGINTEGER', 'FLOAT', 'DOUBLE', 'TEXT', 'DATE', 'DATEHIGHPRECISION', 'DATEONLY', 'TIMEONLY', 'TIMESTAMPOFFSET', 'BLOB', 'GUID', 'RASTER'], Literal['Geometry', 'SmallInteger', 'Integer', 'BigInteger', 'Single', 'Double', 'String', 'Date', 'DateOnly', 'TimeOnly', 'TimestampOffset', 'Guid', 'Raster']] = FrozenDict({'SHORT': 'SmallInteger', 'LONG': 'Integer', 'BIGINTEGER': 'BigInteger', 'FLOAT': 'Single', 'DOUBLE': 'Double', 'TEXT': 'String', 'DATE': 'Date', 'DATEHIGHPRECISION': 'Date', 'DATEONLY': 'DateOnly', 'TIMEONLY': 'TimeOnly', 'TIMESTAMPOFFSET': 'TimestampOffset', 'BLOB': 'Blob', 'GUID': 'Guid', 'RASTER': 'Raster'})

  Mapping of field data type keywords where keys are those used in field creation and values are the corresponding keywords found in `arcpy.Field.type` (and `config.yml`).

FIELD_TYPE_KEYWORDS: FrozenDict[Literal['Geometry', 'SmallInteger', 'Integer', 'BigInteger', 'Single', 'Double', 'String', 'Date', 'DateOnly', 'TimeOnly', 'TimestampOffset', 'Guid', 'Raster'], Literal['SHORT', 'LONG', 'BIGINTEGER', 'FLOAT', 'DOUBLE', 'TEXT', 'DATE', 'DATEHIGHPRECISION', 'DATEONLY', 'TIMEONLY', 'TIMESTAMPOFFSET', 'BLOB', 'GUID', 'RASTER']] = FrozenDict({'SmallInteger': 'SHORT', 'Integer': 'LONG', 'BigInteger': 'BIGINTEGER', 'Single': 'FLOAT', 'Double': 'DOUBLE', 'String': 'TEXT', 'Date': 'DATEHIGHPRECISION', 'DateOnly': 'DATEONLY', 'TimeOnly': 'TIMEONLY', 'TimestampOffset': 'TIMESTAMPOFFSET', 'Blob': 'BLOB', 'Guid': 'GUID', 'Raster': 'RASTER'})

  Mapping of field data type keywords where keys are those found in `arcpy.Field.type` (and `config.yml`) and values are the corresponding keywords used in field creation. In this mapping, `Date` is mapped to `DATEHIGHPRECISION`, not `DATE`.

FeatureAttributeErrorCode

  The specific error codes for use with `FeatureAttributeErrorMessage`. These values should be succinct indicators of the type of error that a Toolkit user could look up in an error glossary for more information than can reasonably be provided in the error table.

  alias of `Literal`[‘ERROR:DOMAIN:INVALID_VALUE’, ‘ERROR:GENERAL:INVALID_VALUE’, ‘ERROR:GENERAL:MANDATORY_IS_NULL’, ‘ERROR:GENERAL:MANDATORY_IS_BLANK’, ‘ERROR:GENERAL:NOT_UPPERCASE’, ‘ERROR:GENERAL:UNIQUENESS’, ‘WARNING:GENERAL:LEADING_TRAILING_SPACE’, ‘ERROR:NGUID:FORMAT’, ‘ERROR:NGUID:V2_FORMAT’, ‘ERROR:NGUID:AGENCY’, ‘ERROR:NGUID:LAYER’, ‘ERROR:NGUID:DUPLICATE’, ‘ERROR:ADDRESS:DUPLICATE’, ‘ERROR:ADDRESS_RANGE:OVERLAP’, ‘ERROR:ADDRESS_RANGE:DECREASING’, ‘ERROR:ROAD_ESN:DEVIATION’, ‘ERROR:ROAD_ESN:CROSSING’, ‘ERROR:ROAD_ESN:OUT_OF_BOUNDS’, ‘ERROR:PARITY:EXPECTED_ZERO’, ‘ERROR:PARITY:EXPECTED_NONZERO’, ‘ERROR:PARITY:MISMATCH’, ‘ERROR:PARITY:INVALID’, ‘ERROR:PARITY:NULL’, ‘ERROR:LEGACY:MISMATCH’, ‘ERROR:GEOCODE:UNKNOWN_MATCH’, ‘ERROR:GEOCODE:WRONG_SIDE’, ‘ERROR:GEOCODE:BOTH_SIDES’, ‘ERROR:GEOCODE:WRONG_COMMUNITY’, ‘ERROR:GEOCODE:OUT_OF_RANGE’, ‘ERROR:GEOCODE:NAME_MISMATCH’, ‘ERROR:CONSISTENCY:ADDRESS_ESN’, ‘ERROR:CONSISTENCY:ROAD_ESN’, ‘ERROR:CONSISTENCY:COMMUNITY’, ‘ERROR:MSAG:NO_ROAD_MATCH’, ‘WARNING:CONSISTENCY:ROAD_LEVEL’, ‘ERROR:GEOMETRY:TOPOLOGY’, ‘WARNING:GEOMETRY:CUTBACK’, ‘WARNING:GEOMETRY:SHORT_SEGMENT’, ‘NOTICE:CONSISTENCY:ROAD_ESN’]

GDBErrorCode

  The specific error codes for use with `GDBErrorMessage`. These values should be succinct indicators of the type of error that a Toolkit user could look up in an error glossary for more information than can reasonably be provided in the error table.

  alias of `Literal`[‘ERROR:PYTHON:EXCEPTION’, ‘ERROR:GDB:MISSING_REQUIRED_DATASET’, ‘ERROR:GDB:MISSING_REQUIRED_FEATURE_CLASS’, ‘ERROR:GDB:EXTRA_ITEM’, ‘ERROR:GDB:MISSING_DOMAIN’, ‘ERROR:GDB:EXTRA_DOMAIN’, ‘ERROR:GDB:INCORRECT_DOMAIN_TYPE’, ‘ERROR:GDB:DOMAIN_MISSING_CODE’, ‘ERROR:GDB:DOMAIN_EXTRA_CODE’, ‘ERROR:GDB:DOMAIN_CODE_VALUE_MISMATCH’, ‘ERROR:GDB:INCORRECT_DOMAIN_DESCRIPTION’, ‘ERROR:DATASET:INCORRECT_SPATIAL_REFERENCE’, ‘ERROR:DATASET:MISSING_TOPOLOGY’, ‘ERROR:DATASET:INCORRECT_TOPOLOGY’, ‘ERROR:DATASET:TOPOLOGY_VIOLATION’, ‘ERROR:FEATURE_CLASS:MISSING_REQUIRED_FIELD’, ‘ERROR:FEATURE_CLASS:EXTRA_FIELD’, ‘ERROR:FEATURE_CLASS:INCORRECT_SPATIAL_REFERENCE’, ‘ERROR:FEATURE_CLASS:INCORRECT_GEOMETRY_TYPE’, ‘ERROR:FEATURE_CLASS:INCORRECT_FEATURE_TYPE’, ‘ERROR:FEATURE_CLASS:EMPTY’, ‘ERROR:FEATURE_CLASS:EMPTY_SUBMISSION’, ‘ERROR:FIELD:INCORRECT_FIELD_TYPE’, ‘ERROR:FIELD:INCORRECT_FIELD_LENGTH’, ‘ERROR:FIELD:INCORRECT_FIELD_DOMAIN’, ‘NOTICE:GDB:MISSING_OPTIONAL_DATASET’, ‘NOTICE:GDB:MISSING_OPTIONAL_FEATURE_CLASS’]

Severity

  The three severity levels of `ValidationErrorMessage` instances.

  alias of `Literal`[‘Notice’, ‘Warning’, ‘Error’]

ValidationCategory

  The categories under which validation routines are grouped.

  alias of `Literal`[‘Geodatabase’, ‘General Feature Class’, ‘Address Point’, ‘Road Centerline’]

parity_dtype = CategoricalDtype(categories=['ODD', 'EVEN', 'BOTH', 'ZERO'], ordered=False)

  An instance of `pandas.CategoricalDtype` based on the members of `Parity`