API Reference

DiffSync front-end classes and logic.

class diffsync.DiffSync(name=None, internal_storage_engine=<class 'diffsync.store.local.LocalStore'>)

Bases: object

Class for storing a group of DiffSyncModel instances and diffing/synchronizing to another DiffSync instance.

__init__(name=None, internal_storage_engine=<class 'diffsync.store.local.LocalStore'>)

Generic initialization function.

Subclasses should be careful to call super().__init__() if they override this method.

add(obj: diffsync.DiffSyncModel)

Add a DiffSyncModel object to the store.

Parameters

obj (DiffSyncModel) – Object to store

Raises

ObjectAlreadyExists – if a different object with the same uid is already present.

count(model: Optional[Union[str, diffsync.DiffSyncModel, Type[diffsync.DiffSyncModel]]] = None)

Count how many objects of one model type exist in the backend store.

Parameters

model (DiffSyncModel) – The DiffSyncModel to check the number of elements. If not provided, default to all.

Returns

Number of elements of the model type

Return type

Int

dict(exclude_defaults: bool = True, **kwargs) → Mapping

Represent the DiffSync contents as a dict, as if it were a Pydantic model.

diff_from(source: diffsync.DiffSync, diff_class: typing.Type[diffsync.diff.Diff] = <class 'diffsync.diff.Diff'>, flags: diffsync.enum.DiffSyncFlags = DiffSyncFlags.NONE, callback: typing.Optional[typing.Callable[[str, int, int], None]] = None) → diffsync.diff.Diff

Generate a Diff describing the difference from the other DiffSync to this one.

Parameters

• source (DiffSync) – Object to diff against.

• diff_class (class) – Diff or subclass thereof to use for diff calculation and storage.

• flags (DiffSyncFlags) – Flags influencing the behavior of this diff operation.

• callback (function) – Function with parameters (stage, current, total), to be called at intervals as the calculation of the diff proceeds.

diff_to(target: diffsync.DiffSync, diff_class: typing.Type[diffsync.diff.Diff] = <class 'diffsync.diff.Diff'>, flags: diffsync.enum.DiffSyncFlags = DiffSyncFlags.NONE, callback: typing.Optional[typing.Callable[[str, int, int], None]] = None) → diffsync.diff.Diff

Generate a Diff describing the difference from this DiffSync to another one.

Parameters

• target (DiffSync) – Object to diff against.

• diff_class (class) – Diff or subclass thereof to use for diff calculation and storage.

• flags (DiffSyncFlags) – Flags influencing the behavior of this diff operation.

• callback (function) – Function with parameters (stage, current, total), to be called at intervals as the calculation of the diff proceeds.

get(obj: Union[str, diffsync.DiffSyncModel, Type[diffsync.DiffSyncModel]], identifier: Union[str, Mapping]) → diffsync.DiffSyncModel

Get one object from the data store based on its unique id.

Parameters

• obj – DiffSyncModel class or instance, or modelname string, that defines the type of the object to retrieve

• identifier – Unique ID of the object to retrieve, or dict of unique identifier keys/values

Raises

• ValueError – if obj is a str and identifier is a dict (can’t convert dict into a uid str without a model class)

• ObjectNotFound – if the requested object is not present

get_all(obj: Union[str, diffsync.DiffSyncModel, Type[diffsync.DiffSyncModel]]) → List[diffsync.DiffSyncModel]

Get all objects of a given type.

Parameters

obj – DiffSyncModel class or instance, or modelname string, that defines the type of the objects to retrieve

Returns

List of Object

Return type

List[DiffSyncModel]

get_all_model_names()

Get all model names.

Returns

List of model names

Return type

List[str]

get_by_uids(uids: List[str], obj: Union[str, diffsync.DiffSyncModel, Type[diffsync.DiffSyncModel]]) → List[diffsync.DiffSyncModel]

Get multiple objects from the store by their unique IDs/Keys and type.

Parameters

• uids – List of unique id / key identifying object in the database.

• obj – DiffSyncModel class or instance, or modelname string, that defines the type of the objects to retrieve

Raises

ObjectNotFound – if any of the requested UIDs are not found in the store

get_or_add_model_instance(obj: diffsync.DiffSyncModel) → Tuple[diffsync.DiffSyncModel, bool]

Attempt to get the object with provided obj identifiers or instantiate obj.

Parameters

obj – An obj of the DiffSyncModel to get or add.

Returns

Provides the existing or new object and whether it was created or not.

get_or_instantiate(model: Type[diffsync.DiffSyncModel], ids: Dict, attrs: Optional[Dict] = None) → Tuple[diffsync.DiffSyncModel, bool]

Attempt to get the object with provided identifiers or instantiate it with provided identifiers and attrs.

Parameters

• model (DiffSyncModel) – The DiffSyncModel to get or create.

• ids (Mapping) – Identifiers for the DiffSyncModel to get or create with.

• attrs (Mapping, optional) – Attributes when creating an object if it doesn’t exist. Defaults to None.

Returns

Provides the existing or new object and whether it was created or not.

Return type

Tuple[DiffSyncModel, bool]

get_or_none(obj: Union[str, diffsync.DiffSyncModel, Type[diffsync.DiffSyncModel]], identifier: Union[str, Mapping]) → Optional[diffsync.DiffSyncModel]

Get one object from the data store based on its unique id or get a None

Parameters

• obj – DiffSyncModel class or instance, or modelname string, that defines the type of the object to retrieve

• identifier – Unique ID of the object to retrieve, or dict of unique identifier keys/values

Raises

ValueError – if obj is a str and identifier is a dict (can’t convert dict into a uid str without a model class)

Returns

DiffSyncModel matching provided criteria

classmethod get_tree_traversal(as_dict: bool = False) → Union[str, Mapping]

Get a string describing the tree traversal for the diffsync object.

Parameters

as_dict – Whether or not to return as a dictionary

Returns

A string or dictionary representation of tree

load()

Load all desired data from whatever backend data source into this instance.

load_from_dict(data: Dict)

The reverse of dict method, taking a dictionary and loading into the inventory.

Parameters

data – Dictionary in the format that dict would export as

remove(obj: diffsync.DiffSyncModel, remove_children: bool = False)

Remove a DiffSyncModel object from the store.

Parameters

• obj (DiffSyncModel) – object to remove

• remove_children (bool) – If True, also recursively remove any children of this object

Raises

ObjectNotFound – if the object is not present

str(indent: int = 0) → str

Build a detailed string representation of this DiffSync.

sync_complete(source: diffsync.DiffSync, diff: diffsync.diff.Diff, flags: diffsync.enum.DiffSyncFlags = DiffSyncFlags.NONE, logger: Optional[structlog._generic.BoundLogger] = None)

Callback triggered after a sync_from operation has completed and updated the model data of this instance.

Note that this callback is only triggered if the sync actually resulted in data changes. If there are no detected changes, this callback will not be called.

The default implementation does nothing, but a subclass could use this, for example, to perform bulk updates to a backend (such as a file) that doesn’t readily support incremental updates to individual records.

Parameters

• source – The DiffSync whose data was used to update this instance.

• diff – The Diff calculated prior to the sync operation.

• flags – Any flags that influenced the sync.

• logger – Logging context for the sync.

sync_from(source: diffsync.DiffSync, diff_class: typing.Type[diffsync.diff.Diff] = <class 'diffsync.diff.Diff'>, flags: diffsync.enum.DiffSyncFlags = DiffSyncFlags.NONE, callback: typing.Optional[typing.Callable[[str, int, int], None]] = None, diff: typing.Optional[diffsync.diff.Diff] = None) → diffsync.diff.Diff

Synchronize data from the given source DiffSync object into the current DiffSync object.

Parameters

• source (DiffSync) – object to sync data from into this one

• diff_class (class) – Diff or subclass thereof to use to calculate the diffs to use for synchronization

• flags (DiffSyncFlags) – Flags influencing the behavior of this sync.

• callback (function) – Function with parameters (stage, current, total), to be called at intervals as the calculation of the diff and subsequent sync proceed.

• diff (Diff) – An existing diff to be used rather than generating a completely new diff.

Returns

Diff between origin object and source

Return type

Diff

Raises

DiffClassMismatch – The provided diff’s class does not match the diff_class

sync_to(target: diffsync.DiffSync, diff_class: typing.Type[diffsync.diff.Diff] = <class 'diffsync.diff.Diff'>, flags: diffsync.enum.DiffSyncFlags = DiffSyncFlags.NONE, callback: typing.Optional[typing.Callable[[str, int, int], None]] = None, diff: typing.Optional[diffsync.diff.Diff] = None) → diffsync.diff.Diff

Synchronize data from the current DiffSync object into the given target DiffSync object.

Parameters

• target (DiffSync) – object to sync data into from this one.

• diff_class (class) – Diff or subclass thereof to use to calculate the diffs to use for synchronization

• flags (DiffSyncFlags) – Flags influencing the behavior of this sync.

• callback (function) – Function with parameters (stage, current, total), to be called at intervals as the calculation of the diff and subsequent sync proceed.

• diff (Diff) – An existing diff that will be used when determining what needs to be synced.

Returns

Diff between origin object and target

Return type

Diff

Raises

DiffClassMismatch – The provided diff’s class does not match the diff_class

top_level: ClassVar[List[str]] = []

List of top-level modelnames to begin from when diffing or synchronizing.

type: ClassVar[Optional[str]] = None

Type of the object, will default to the name of the class if not provided.

update(obj: diffsync.DiffSyncModel)

Update a DiffSyncModel object to the store.

Parameters

obj (DiffSyncModel) – Object to store

Raises

ObjectAlreadyExists – if a different object with the same uid is already present.

update_or_add_model_instance(obj: diffsync.DiffSyncModel) → Tuple[diffsync.DiffSyncModel, bool]

Attempt to update an existing object with provided obj ids/attrs or instantiate obj.

Parameters

instance – An instance of the DiffSyncModel to update or create.

Returns

Provides the existing or new object and whether it was created or not.

update_or_instantiate(model: Type[diffsync.DiffSyncModel], ids: Dict, attrs: Dict) → Tuple[diffsync.DiffSyncModel, bool]

Attempt to update an existing object with provided ids/attrs or instantiate it with provided identifiers and attrs.

Parameters

• model (DiffSyncModel) – The DiffSyncModel to update or create.

• ids (Dict) – Identifiers for the DiffSyncModel to update or create with.

• attrs (Dict) – Attributes when creating/updating an object if it doesn’t exist. Pass in empty dict, if no specific attrs.

Returns

Provides the existing or new object and whether it was created or not.

Return type

Tuple[DiffSyncModel, bool]

class diffsync.DiffSyncModel(*, model_flags: diffsync.enum.DiffSyncModelFlags = DiffSyncModelFlags.NONE, diffsync: Optional[DiffSync] = None)

Bases: pydantic.main.BaseModel

Base class for all DiffSync object models.

Note that read-only APIs of this class are implemented as get_*() functions rather than as properties; this is intentional as specific model classes may want to use these names (type, keys, attrs, etc.) as model attributes and we want to avoid any ambiguity or collisions.

This class has several underscore-prefixed class variables that subclasses should set as desired; see below.

NOTE: The groupings _identifiers, _attributes, and _children are mutually exclusive; any given field name can be included in at most one of these three tuples.

class Config

Bases: object

Pydantic class configuration.

arbitrary_types_allowed = True

add_child(child: diffsync.DiffSyncModel)

Add a child reference to an object.

The child object isn’t stored, only its unique id. The name of the target attribute is defined in _children per object type

Raises

• ObjectStoreWrongType – if the type is not part of _children

• ObjectAlreadyExists – if the unique id is already stored

classmethod create(diffsync: diffsync.DiffSync, ids: Mapping, attrs: Mapping) → Optional[diffsync.DiffSyncModel]

Instantiate this class, along with any platform-specific data creation.

Subclasses must call super().create() or self.create_base(); they may wish to then override the default status information by calling set_status() to provide more context (such as details of any interactions with underlying systems).

Parameters

• diffsync – The master data store for other DiffSyncModel instances that we might need to reference

• ids – Dictionary of unique-identifiers needed to create the new object

• attrs – Dictionary of additional attributes to set on the new object

Returns

instance of this class, if all data was successfully created. None: if data creation failed in such a way that child objects of this model should not be created.

Return type

DiffSyncModel

Raises

ObjectNotCreated – if an error occurred.

classmethod create_base(diffsync: diffsync.DiffSync, ids: Mapping, attrs: Mapping) → Optional[diffsync.DiffSyncModel]

Instantiate this class, along with any platform-specific data creation.

This method is not meant to be subclassed, users should redefine create() instead.

Parameters

• diffsync – The master data store for other DiffSyncModel instances that we might need to reference

• ids – Dictionary of unique-identifiers needed to create the new object

• attrs – Dictionary of additional attributes to set on the new object

Returns

instance of this class.

Return type

DiffSyncModel

classmethod create_unique_id(**identifiers) → str

Construct a unique identifier for this model class.

Parameters

**identifiers – Dict of identifiers and their values, as in get_identifiers().

delete() → Optional[diffsync.DiffSyncModel]

Delete any platform-specific data corresponding to this instance.

Subclasses must call super().delete() or self.delete_base(); they may wish to then override the default status information by calling set_status() to provide more context (such as details of any interactions with underlying systems).

Returns

this instance, if all data was successfully deleted. None: if data deletion failed in such a way that child objects of this model should not be deleted.

Return type

DiffSyncModel

Raises

ObjectNotDeleted – if an error occurred.

delete_base() → Optional[diffsync.DiffSyncModel]

Base delete method Delete any platform-specific data corresponding to this instance.

This method is not meant to be subclassed, users should redefine delete() instead.

Returns

this instance.

Return type

DiffSyncModel

dict(**kwargs) → dict

Convert this DiffSyncModel to a dict, excluding the diffsync field by default as it is not serializable.

diffsync: Optional[diffsync.DiffSync]

the DiffSync instance that owns this model instance.

Type

Optional

get_attrs() → Mapping

Get all the non-primary-key attributes or parameters for this object.

Similar to Pydantic’s BaseModel.dict() method, with the following key differences: 1. Does not include the fields in _identifiers 2. Only includes fields explicitly listed in _attributes 3. Does not include any additional fields not listed in _attributes

Returns

Dictionary of attributes for this object

Return type

dict

classmethod get_children_mapping() → Mapping[str, str]

Get the mapping of types to fieldnames for child models of this model.

get_identifiers() → Mapping

Get a dict of all identifiers (primary keys) and their values for this object.

Returns

dictionary containing all primary keys for this device, as defined in _identifiers

Return type

dict

get_shortname() → str

Get the (not guaranteed-unique) shortname of an object, if any.

By default the shortname is built based on all the keys defined in _shortname. If _shortname is not specified, then this function is equivalent to get_unique_id().

Returns

Shortname of this object

Return type

str

get_status() → Tuple[diffsync.enum.DiffSyncStatus, str]

Get the status of the last create/update/delete operation on this object, and any associated message.

classmethod get_type() → str

Return the type AKA modelname of the object or the class

Returns

modelname of the class, used in to store all objects

Return type

str

get_unique_id() → str

Get the unique ID of an object.

By default the unique ID is built based on all the primary keys defined in _identifiers.

Returns

Unique ID for this object

Return type

str

json(**kwargs) → str

Convert this DiffSyncModel to a JSON string, excluding the diffsync field by default as it is not serializable.

model_flags: diffsync.enum.DiffSyncModelFlags

any non-default behavioral flags for this DiffSyncModel.

Can be set as a class attribute or an instance attribute as needed.

Type

Optional

remove_child(child: diffsync.DiffSyncModel)

Remove a child reference from an object.

The name of the storage attribute is defined in _children per object type.

Raises

• ObjectStoreWrongType – if the child model type is not part of _children

• ObjectNotFound – if the child wasn’t previously present.

set_status(status: diffsync.enum.DiffSyncStatus, message: str = '')

Update the status (and optionally status message) of this model in response to a create/update/delete call.

str(include_children: bool = True, indent: int = 0) → str

Build a detailed string representation of this DiffSyncModel and optionally its children.

update(attrs: Mapping) → Optional[diffsync.DiffSyncModel]

Update the attributes of this instance, along with any platform-specific data updates.

Subclasses must call super().update() or self.update_base(); they may wish to then override the default status information by calling set_status() to provide more context (such as details of any interactions with underlying systems).

Parameters

attrs – Dictionary of attributes to update on the object

Returns

this instance, if all data was successfully updated. None: if data updates failed in such a way that child objects of this model should not be modified.

Return type

DiffSyncModel

Raises

ObjectNotUpdated – if an error occurred.

update_base(attrs: Mapping) → Optional[diffsync.DiffSyncModel]

Base Update method to update the attributes of this instance, along with any platform-specific data updates.

This method is not meant to be subclassed, users should redefine update() instead.

Parameters

attrs – Dictionary of attributes to update on the object

Returns

this instance.

Return type

DiffSyncModel

• diffsync.diff
• diffsync.enum
• diffsync.exceptions
• diffsync.helpers
• diffsync.logging
• diffsync.utils

Subpackages

• API Reference
  • diffsync.store.local
  • diffsync.store.redis