[This is preliminary documentation and is subject to change.]
Facilitates working with large-scale distributed database systems like the ones used by Google, Facebook, Twitter et. al.
Classes
Class | Description | |
---|---|---|
![]() | Command |
Defines a command sent into an IDistributedDataStore implementor to retrieve or change(if supported) data.
A Command is a named bag of paremeters where every parameter has a name and a value.
Every command has a unique Identity(GUID) which represents a token of the whole command state (name, type,all params).
The identity is used for quick lookup/caching. The identity may be supplied externally as
business code may map certain parameters into GUID and later reuse the same GUID to retrieve the cached command result, for example
a web server app may cache command "GetPurchases(user=123, year=2015, month=3)" under session key "MY_PURCHASES_201503" to later
retrieve a cached (if available) command results from the DB layer, this way the DB server does not have to store the whole
commands with all params as the cache key (which would have been slow to compare and would have induced GC pressure).
Warning: DO NOT CACHE command identity value on a client (i.e. web page) in an un-encrypted state, as this is a security flaw
|
![]() | Command Param |
Represents a distributed command parameter
|
![]() | DataParcelAttribute |
Decorates Pacel-derivative classes specifying distributed data store options.
Unlike the CRUD family of metadata attributes this attributed is NOT TARGETABLE on purpose
beacause different sharding definitions would have affected the properties of the parcel which could have been
very complex to maintain/account for. So, every parcel has ONLY ONE set opf metadata definition.
In case when different parcel definitions needed a new parcel type should be created which can reuse the payload - this is much
easier to implement (two parcels) than targeting within the same parcel.
Table mappings are targetable
|
![]() | DistributedDataAccessException |
Thrown by distributed data access classes
|
![]() | DistributedDataParcelSerializationException |
Thrown by distributed data access classes when parcel serialization problems happen
|
![]() | GDIDRangeComparer |
Compares GDID regardless of authority. This is useful for range checking, when authorities generating GDIDs in the same
range should be disregarded. Use GDIDRangeComparer.Instance.
Only relative range comparison can be made.
The Equality returned by this comparer can not be relied upon for GDID comparison as it disregards authority.
Equality can only be tested for range comparison.
|
![]() | NullCommandResult |
Represents a result of command that is absent. This is needed to distinguish from null reference
|
![]() | Parcel |
Describes a data parcel - a piece of logically-grouped data that gets fetched from/comitted into a distributed backend system.
Parcels represent an atomic unit of change, a changeset that gets replicated between failover hosts.
Every parcel has a Payload property that stores business data of interest that the parcel contains.
This class is not thread-safe. Use DeepClone() to create 100% copies for working in multiple threads.
This particular class serves as a very base for all Parcel implementations
|
![]() | Parcel TPayload |
Describes a data parcel - a piece of logically-grouped data that gets fetched from/comitted into a distributed backend system.
Parcels represent an atomic unit of change, a changeset that gets replicated between failover hosts.
Every parcel has a Payload property that stores business data of interest that the parcel contains.
This class is not thread-safe.
This particular class serves as a very base for distributed data store implementations
|
![]() | ParcelSealValidationException |
Thrown by Parcel.Seal() method trying to ensure parcel consistency before it gets sealed
|
![]() | ParcelValidationException |
Thrown by distributed data access classes to indicate some data validation error
|
Structures
Structure | Description | |
---|---|---|
![]() | CachePolicyData |
Holds data per IParcelCachePolicy contract
|
![]() | GDID |
Represents a Global Distributed ID key (key field) used in distributed databases that identify entities with a combination of unsigned 32 bit integer
'Era' and unsigned 64 bit integer 'ID'. The first 32 bit integer is an 'era' in which the 'ID' (64 bit) was created, consequently
a GDID is a 12 byte = 96 bit integer that can hold 2^96 = 79,228,162,514,264,337,593,543,950,336 combinations.
The ID consists of two segments: 4 bit authority + 60 bits counter. Authority segment occupies the most significant 4 bits of uint64, so
the system may efficiently query the data store to identify the highest stored ID value in a range.
Authorities identify one of 16 possible ID generation sources in the global distributed system, therefore ID duplications are not
possible between authorities.
Within a single era, GDID structure may identify 2^60 = 1,152,921,504,606,846,976(per authority) * 16(authorities) = 2^64 = 18,446,744,073,709,551,616 total combinations.
Because of such a large number of combinations supported by GDID.ID alone (having the same Era), some systems may always use Era=0 and only store the ID part
(i.e. as UNSIGNED BIGINT in SQL datastores).
Note GDID.Zero is never returned by generators as it represents the absence of a value
|
![]() | GDIDSymbol |
Represents a tuple of GDID and its symbolic representation (framework usualy uses an ELink as symbolic representation).
This struct is needed to pass GDID along with its ELink representation together.
Keep in mind that string poses a GC load, so this stuct is not suitable for beiing used as a pile cache key
|
![]() | ShardingPointer |
Contains information about sharding parcel type and ID
which is used to calculate the physical location of parcel data
|
![]() | TargetTableMapping |
Provides information about targetname->table name, sequence name(and possibly other) mappings
|
Interfaces
Interface | Description | |
---|---|---|
![]() | IArea |
Provides information about an area of a bank schema. This information does not depend on a particular bank instance,
as it is common for all banks that implement the same schema.
Area provides configuration information for parcels that it can store.
Every instance of this (interface-implementer) class has a corresponding IAreaInstance instance that stores information
for every particular bank, i.e. what distribution policies are applied (such as sharding) to the parcels stored in this area
|
![]() | IAreaInstance |
Represents data for a concrete instance of schema area within banks, this depends on a concrete bank instance (unlike it's complementary part IArea)
|
![]() | IBank |
Provides abstraction for Global Database Bank instance
|
![]() | ICachePolicy | |
![]() | IDistributedDataStore |
Represents a data store that works with large distributed systems that support OLTP-style processing
and provide data querying, caching, partitioning/sharding, failover and replication.
These systems are designed to handle billions of rows that need to be accessed by millions of active concurrent users,
so the design is specific to this model that scales horizontally. There is no need to use this technology for medium and smaller data stores
as it imposes specific requirements on how application is written/interacts with the backend system. This technology is based on the idea
of Parcels - an atomic unit of data change. Parcels get replicated between hosts for failover and performance reasons.
Note:
NFX library does not provide the implementation for this technology, only marker interfaces so developers can plan for distributed backends
in future
|
![]() | IDistributedDataStoreImplementation | |
![]() | IDistributedStableHashProvider |
Denotes entities that provide ULONG STABLE hash code for use in a distributed (large scale) system.
This is needed primarily for cluster/large datasets to properly compute 64bit sharding addresses and to differentiate
from GetHashCode() that returns 32 bits unstable hash for local object location in hashtables.
DO not confuse with object.GetHashCode() which is un-suitable for long-term persistence
|
![]() | IReplicatable |
Denotes a distributed entity that can be replicated to a different location/site/instance.
Normally only Parcel and its derivatives should implement this interface
|
![]() | IReplicationVersionInfo | |
![]() | ISchema |
Provides information about schema of data store banks. Schema defines areas of the bank, where every area
defines what parcel types can be stored. Each bank implements only one bank schema
|
![]() | IShardingPointerProvider |
Denotes an entity which provides a sharding parcel type along with sharding ID (ShardingPointer) that can be used to determine
data location via conversion of this id into physical shard #(particular server) that this entity represents
|
Enumerations
Enumeration | Description | |
---|---|---|
![]() | DataCaching |
Stipulates where data gets cached
|
![]() | DataVeracity |
Stipulates levels of data fidality, the higher the level - the more accurately verified data is provided
|
![]() | ParcelState |
Represents data parcel states - creating, modifying, sealed
|