Core functionalities

Computation Graph

The ComputationGraph is the workhorse of dynet. From the Dynet technical report :

[The] computation graph represents symbolic computation, and the results of the computation are evaluated lazily: the computation is only performed once the user explicitly asks for it (at which point a “forward” computation is triggered). Expressions that evaluate to scalars (i.e. loss values) can also be used to trigger a “backward” computation, computing the gradients of the computation with respect to the parameters.
int dynet::get_number_of_active_graphs()

Gets the number of active graphs.

This is 0 or 1, you can’t create more than one graph at once

Return
Number of active graphs

unsigned dynet::get_current_graph_id()

Get id of the current active graph.

This can help check whether a graph is stale

Return
Id of the current graph

struct dynet::ComputationGraph
#include <dynet.h>

Computation graph where nodes represent forward and backward intermediate values, and edges represent functions of multiple values.

To represent the fact that a function may have multiple arguments, edges have a single head and 0, 1, 2, or more tails. (Constants, inputs, and parameters are represented as functions of 0 parameters.) Example: given the function z = f(x, y), z, x, and y are nodes, and there is an edge representing f with which points to the z node (i.e., its head), and x and y are the tails of the edge. You shouldn’t need to use most methods from the ComputationGraph except for backward since most of them are available directly from the Expression class.

Public Functions

dynet::ComputationGraphComputationGraph()

Default constructor.

VariableIndex dynet::ComputationGraphadd_input(real s, Device *device)

Add scalar input.

The computational network will pull inputs in from the user’s data structures and make them available to the computation

Return
The index of the created variable
Parameters
  • s: Real number
  • device: The device to place input value

VariableIndex dynet::ComputationGraphadd_input(const real *ps, Device *device)

Add scalar input by pointer.

The computational network will pull inputs in from the user’s data structures and make them available to the computation

Return
The index of the created variable
Parameters
  • ps: Pointer to a real number
  • device: The device to place input value

VariableIndex dynet::ComputationGraphadd_input(const Dim &d, const std::vector<float> &data, Device *device)

Add multidimentsional input.

The computational network will pull inputs in from the user’s data structures and make them available to the computation

Return
The index of the created variable
Parameters
  • d: Desired shape of the input
  • data: Input data (as a 1 dimensional array)
  • data: The data points corresponding to each index
  • device: The device to place input value

VariableIndex dynet::ComputationGraphadd_input(const Dim &d, const std::vector<float> *pdata, Device *device)

Add multidimentsional input by pointer.

The computational network will pull inputs in from the user’s data structures and make them available to the computation

Return
The index of the created variable
Parameters
  • d: Desired shape of the input
  • pdata: Pointer to the input data (as a 1 dimensional array)
  • device: The device to place input value

VariableIndex dynet::ComputationGraphadd_input(const Dim &d, const std::vector<unsigned int> &ids, const std::vector<float> &data, Device *device, float defdata = 0.f)

Add sparse input.

The computational network will pull inputs in from the user’s data structures and make them available to the computation. Represents specified (not learned) inputs to the network in sparse array format, with an optional default value.

Return
The index of the created variable
Parameters
  • d: Desired shape of the input
  • ids: The indexes of the data points to update
  • data: The data points corresponding to each index
  • device: The device to place input value
  • defdata: The default data with which to set the unspecified data points

VariableIndex dynet::ComputationGraphadd_parameters(Parameter p)

Add a parameter to the computation graph.

Return
The index of the created variable
Parameters

VariableIndex dynet::ComputationGraphadd_parameters(LookupParameter p)

Add a full matrix of lookup parameters to the computation graph.

Return
The index of the created variable
Parameters

VariableIndex dynet::ComputationGraphadd_const_parameters(Parameter p)

Add a parameter to the computation graph (but don’t update)

Return
The index of the created variable
Parameters

VariableIndex dynet::ComputationGraphadd_const_parameters(LookupParameter p)

Add a full matrix of lookup parameter to the computation graph (but don’t update)

Return
The index of the created variable
Parameters

VariableIndex dynet::ComputationGraphadd_lookup(LookupParameter p, const unsigned *pindex)

Add a lookup parameter to the computation graph.

Use pindex to point to a memory location where the index will live that the caller owns

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • pindex: Pointer to the index to lookup

VariableIndex dynet::ComputationGraphadd_lookup(LookupParameter p, unsigned index)

Add a lookup parameter to the computation graph.

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • index: Index to lookup

VariableIndex dynet::ComputationGraphadd_lookup(LookupParameter p, const std::vector<unsigned> *pindices)

Add lookup parameters to the computation graph.

Use pindices to point to a memory location where the indices will live that the caller owns

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • pindices: Pointer to the indices to lookup

VariableIndex dynet::ComputationGraphadd_lookup(LookupParameter p, const std::vector<unsigned> &indices)

Add lookup parameters to the computation graph.

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • indices: Indices to lookup

VariableIndex dynet::ComputationGraphadd_const_lookup(LookupParameter p, const unsigned *pindex)

Add a lookup parameter to the computation graph.

Just like add_lookup, but don’t optimize the lookup parameters

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • pindex: Pointer to the indices to lookup

VariableIndex dynet::ComputationGraphadd_const_lookup(LookupParameter p, unsigned index)

Add a lookup parameter to the computation graph.

Just like add_lookup, but don’t optimize the lookup parameters

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • index: Index to lookup

VariableIndex dynet::ComputationGraphadd_const_lookup(LookupParameter p, const std::vector<unsigned> *pindices)

Add lookup parameters to the computation graph.

Just like add_lookup, but don’t optimize the lookup parameters

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • pindices: Pointer to the indices to lookup

VariableIndex dynet::ComputationGraphadd_const_lookup(LookupParameter p, const std::vector<unsigned> &indices)

Add lookup parameters to the computation graph.

Just like add_lookup, but don’t optimize the lookup parameters

Return
The index of the created variable
Parameters
  • p: Lookup parameter from which to pick
  • indices: Indices to lookup

template <class Function>
VariableIndex dynet::ComputationGraphadd_function(const std::initializer_list<VariableIndex> &arguments)

Add a function to the computation graph.

This what is called when creating an expression

Return
The index of the output variable
Parameters
  • arguments: List of the arguments indices
Template Parameters
  • Function: Function to be applied

template <class Function, typename… Args>
VariableIndex dynet::ComputationGraphadd_function(const std::initializer_list<VariableIndex> &arguments, Args&&... side_information)

Add a function to the computation graph (with side information)

This what is called when creating an expression

Return
The index of the output variable
Parameters
  • arguments: List of the arguments indices
  • side_information: Side information that is needed to compute the function
Template Parameters
  • Function: Function to be applied

void dynet::ComputationGraphclear()

Reset ComputationGraph to a newly created state.

[long description]

void dynet::ComputationGraphcheckpoint()

Set a checkpoint.

void dynet::ComputationGraphrevert()

Revert to last checkpoint.

Dim &dynet::ComputationGraphget_dimension(VariableIndex index) const

Get dimension of a node.

Return
Dimension
Parameters
  • index: Variable index of the node

const Tensor &dynet::ComputationGraphforward(const Expression &last)

Run complete forward pass from first node to given one, ignoring all precomputed values.

Return
Value of the last Expression after execution
Parameters
  • last: Expression up to which the forward pass must be computed

const Tensor &dynet::ComputationGraphforward(VariableIndex i)

Run complete forward pass from first node to given one, ignoring all precomputed values.

Return
Value of the end Node after execution
Parameters
  • i: Variable index of the node up to which the forward pass must be computed

const Tensor &dynet::ComputationGraphincremental_forward(const Expression &last)

Run forward pass from the last computed node to given one.

Useful if you want to add nodes and evaluate just the new parts.

Return
Value of the last Expression after execution
Parameters
  • last: Expression up to which the forward pass must be computed

const Tensor &dynet::ComputationGraphincremental_forward(VariableIndex i)

Run forward pass from the last computed node to given one.

Useful if you want to add nodes and evaluate just the new parts.

Return
Value of the end Node after execution
Parameters
  • last: Variable index of the node up to which the forward pass must be computed

const Tensor &dynet::ComputationGraphget_value(VariableIndex i)

Get forward value for node at index i.

Performs forward evaluation if note available (may compute more than strictly what is needed).

Return
Requested value
Parameters
  • i: Index of the variable from which you want the value

const Tensor &dynet::ComputationGraphget_value(const Expression &e)

Get forward value for the given expression.

Performs forward evaluation if note available (may compute more than strictly what is needed).

Return
Requested value
Parameters

const Tensor &dynet::ComputationGraphget_gradient(VariableIndex i)

Get gradient for node at index i.

Performs backward pass if not available (may compute more than strictly what is needed).

Return
Requested gradient
Parameters
  • i: Index of the variable from which you want the gradient

const Tensor &dynet::ComputationGraphget_gradient(const Expression &e)

Get forward gradient for the given expression.

Performs backward pass if not available (may compute more than strictly what is needed).

Return
Requested gradient
Parameters

void dynet::ComputationGraphinvalidate()

Clears forward caches (for get_value etc).

void dynet::ComputationGraphbackward(const Expression &last, bool full = false)

Computes backward gradients from the front-most evaluated node.

The parameter full specifies whether the gradients should be computed for all nodes (true) or only non-constant nodes.

By default, a node is constant unless

  1. it is a parameter node
  2. it depends on a non-constant node

Thus, functions of constants and inputs are considered as constants.

Turn full on if you want to retrieve gradients w.r.t. inputs for instance. By default this is turned off, so that the backward pass ignores nodes which have no influence on gradients w.r.t. parameters for efficiency.

Parameters
  • last: Expression from which to compute the gradient
  • full: Whether to compute all gradients (including with respect to constant nodes).

void dynet::ComputationGraphbackward(VariableIndex i, bool full = false)

Computes backward gradients from node i (assuming it already been evaluated).

The parameter full specifies whether the gradients should be computed for all nodes (true) or only non-constant nodes.

By default, a node is constant unless

  1. it is a parameter node
  2. it depends on a non-constant node

Thus, functions of constants and inputs are considered as constants.

Turn full on if you want to retrieve gradients w.r.t. inputs for instance. By default this is turned off, so that the backward pass ignores nodes which have no influence on gradients w.r.t. parameters for efficiency.

Parameters
  • i: Index of the node from which to compute the gradient
  • full: Whether to compute all gradients (including with respect to constant nodes). Turn this on if you want to retrieve gradients w.r.t. inputs for instance. By default this is turned off, so that the backward pass ignores nodes which have no influence on gradients w.r.t. parameters for efficiency.

void dynet::ComputationGraphprint_graphviz() const

Used for debugging.

unsigned dynet::ComputationGraphget_id() const

Get the unique graph ID.

This ID is incremented by 1 each time a computation graph is created

Return
graph is

Nodes

Nodes are constituents of the computation graph. The end user doesn’t interact with Nodes but with Expressions.

However implementing new operations requires to create a new subclass of the Node class described below.

struct dynet::Node
#include <dynet.h>

Represents an SSA variable.

Contains information on tha computation node : arguments, output value and gradient of the output with respect to the function. This class must be inherited to implement any new operation. See nodes.cc for examples. An operation on expressions can then be created from the new Node, see expr.h/expr.cc for examples

Subclassed by dynet::Abs, dynet::Acos, dynet::Acosh, dynet::AddVectorToAllColumns, dynet::AffineTransform, dynet::Asin, dynet::Asinh, dynet::Atan, dynet::Atanh, dynet::Average, dynet::AverageColumns, dynet::BinaryLogLoss, dynet::BlockDropout, dynet::CircularConvolution, dynet::CircularCorrelation, dynet::Concatenate, dynet::ConcatenateToBatch, dynet::Constant, dynet::ConstantMinusX, dynet::ConstantPlusX, dynet::ConstParameterNode, dynet::ConstrainedSoftmax, dynet::ConstScalarMultiply, dynet::Conv2D, dynet::Cos, dynet::Cosh, dynet::Cube, dynet::CwiseMultiply, dynet::CwiseQuotient, dynet::CwiseSum, dynet::DotProduct, dynet::Dropout, dynet::DropoutBatch, dynet::DropoutDim, dynet::Erf, dynet::Exp, dynet::ExponentialLinearUnit, dynet::Filter1DNarrow, dynet::FlipGradient, dynet::FoldRows, dynet::GaussianNoise, dynet::Hinge, dynet::HingeDim, dynet::HuberDistance, dynet::Identity, dynet::InnerProduct3D_1D, dynet::InnerProduct3D_1D_1D, dynet::InputNode, dynet::KMaxPooling, dynet::KMHNGram, dynet::L1Distance, dynet::L2Norm, dynet::Log, dynet::LogDet, dynet::LogGamma, dynet::LogisticSigmoid, dynet::LogSoftmax, dynet::LogSumExp, dynet::LogSumExpDimension, dynet::MatrixInverse, dynet::MatrixMultiply, dynet::Max, dynet::MaxDimension, dynet::MaxPooling1D, dynet::MaxPooling2D, dynet::Min, dynet::MinDimension, dynet::MomentBatches, dynet::MomentDimension, dynet::MomentElements, dynet::Negate, dynet::NoBackprop, dynet::PairwiseRankLoss, dynet::ParameterNodeBase, dynet::PickBatchElements, dynet::PickElement, dynet::PickNegLogSoftmax, dynet::PickRange, dynet::PoissonRegressionLoss, dynet::Pow, dynet::RandomBernoulli, dynet::RandomGumbel, dynet::RandomNormal, dynet::RandomUniform, dynet::Rectify, dynet::Reshape, dynet::RestrictedLogSoftmax, dynet::ScalarInputNode, dynet::SelectCols, dynet::SelectRows, dynet::SigmoidLinearUnit, dynet::Sin, dynet::Sinh, dynet::Softmax, dynet::SoftSign, dynet::SparseInputNode, dynet::Sparsemax, dynet::SparsemaxLoss, dynet::Sqrt, dynet::Square, dynet::SquaredEuclideanDistance, dynet::SquaredNorm, dynet::StdBatches, dynet::StdDimension, dynet::StdElements, dynet::StridedSelect, dynet::Sum, dynet::SumDimension, dynet::SumElements, dynet::Tan, dynet::Tanh, dynet::ToDevice, dynet::TraceOfProduct, dynet::Transpose, dynet::VanillaLSTMC, dynet::VanillaLSTMGates, dynet::VanillaLSTMH, dynet::WeightNormalization

Public Functions

virtual Dim dynet::Nodedim_forward(const std::vector<Dim> &xs) const = 0

Compute dimensions of result for given dimensions of inputs.

Also checks to make sure inputs are compatible with each other

Return
Dimension of the output
Parameters
  • xs: Vector containing the dimensions of the inputs

virtual std::string dynet::Nodeas_string(const std::vector<std::string> &args) const = 0

Returns important information for debugging.

See nodes-conv.cc for examples

Return
String description of the node
Parameters
  • args: String descriptions of the arguments

size_t dynet::Nodeaux_storage_size() const

Size of the auxiliar storage.

in general, this will return an empty size, but if a component needs to store extra information in the forward pass for use in the backward pass, it can request the memory here (nb. you could put it on the Node object, but in general, edges should not allocate tensor memory since memory is managed centrally for the entire computation graph).

Return
Size

virtual void dynet::Nodeforward_impl(const std::vector<const Tensor *> &xs, Tensor &fx) const = 0

Forward computation.

This function contains the logic for the forward pass. Some implementation remarks from nodes.cc:

  1. fx can be understood as a pointer to the (preallocated) location for the result of forward to be stored
  2. fx is not initialized, so after calling forward fx must point to the correct answer
  3. fx can be repointed to an input, if forward(x) evaluates to x (e.g., in reshaping)
  4. scalars results of forward are placed in fx.v[0]
  5. DYNET manages its own memory, not Eigen, and it is configured with the EIGEN_NO_MALLOC option. If you get an error about Eigen attempting to allocate memory, it is (probably) because of an implicit creation of a temporary variable. To tell Eigen this is not necessary, the noalias() method is available. If you really do need a temporary variable, its capacity must be requested by Node::aux_storage_size

Note on debugging problems with differentiable components

  • fx is uninitialized when forward is called- are you relying on it being 0?

Parameters
  • xs: Pointers to the inputs
  • fx: pointer to the (preallocated) location for the result of forward to be stored

virtual void dynet::Nodebackward_impl(const std::vector<const Tensor *> &xs, const Tensor &fx, const Tensor &dEdf, unsigned i, Tensor &dEdxi) const = 0

Accumulates the derivative of E with respect to the ith argument to f, that is, xs[i].

This function contains the logic for the backward pass. Some implementation remarks from nodes.cc:

  1. dEdxi MUST ACCUMULATE a result since multiple calls to forward may depend on the same x_i. Even, e.g., Identity must be implemented as dEdx1 += dEdf. THIS IS EXTREMELY IMPORTANT
  2. scalars results of forward are placed in fx.v[0]
  3. DYNET manages its own memory, not Eigen, and it is configured with the EIGEN_NO_MALLOC option. If you get an error about Eigen attempting to allocate memory, it is (probably) because of an implicit creation of a temporary variable. To tell Eigen this is not necessary, the noalias() method is available. If you really do need a temporary variable, its capacity must be requested by Node::aux_storage_size

Note on debugging problems with differentiable components

  • dEdxi must accummulate (see point 4 above!)

Parameters
  • xs: Pointers to inputs
  • fx: Output
  • dEdf: Gradient of the objective w.r.t the output of the node
  • i: Index of the input w.r.t which we take the derivative
  • dEdxi: Gradient of the objective w.r.t the input of the node

virtual bool dynet::Nodesupports_multibatch() const

Whether this node supports computing multiple batches in one call.

If true, forward and backward will be called once with a multi-batch tensor. If false, forward and backward will be called multiple times for each item.

Return
Support for multibatch

virtual bool dynet::Nodesupports_multidevice() const

Whether this node supports processing inputs/outputs on multiple devices.

DyNet will throw an error if you try to process inputs and outputs on different devices unless this is activated.

Return
Support for multi-device

void dynet::Nodeforward(const std::vector<const Tensor *> &xs, Tensor &fx) const

perform the forward/backward passes in one or multiple calls

Parameters
  • xs: Pointers to the inputs
  • fx: pointer to the (preallocated) location for the result of forward to be stored

void dynet::Nodebackward(const std::vector<const Tensor *> &xs, const Tensor &fx, const Tensor &dEdf, unsigned i, Tensor &dEdxi) const

perform the backward passes in one or multiple calls

Parameters
  • xs: Pointers to inputs
  • fx: Output
  • dEdf: Gradient of the objective w.r.t the output of the node
  • i: Index of the input w.r.t which we take the derivative
  • dEdxi: Gradient of the objective w.r.t the input of the node

virtual int dynet::Nodeautobatch_sig(const ComputationGraph &cg, SigMap &sm) const

signature for automatic batching This will be equal only for nodes that can be combined. Returns 0 for unbatchable functions.

virtual std::vector<int> dynet::Nodeautobatch_concat(const ComputationGraph &cg) const

which inputs can be batched This will be true for inputs that should be concatenated when autobatching, and false for inputs that should be shared among all batches.

virtual Node *dynet::Nodeautobatch_pseudo_node(const ComputationGraph &cg, const std::vector<VariableIndex> &batch_ids) const

create a pseudonode for autobatching This will combine together multiple nodes into one big node for the automatic batching functionality. When a node representing one component of the mini-batch can be used as-is it is OK to just return the null pointer, otherwise we should make the appropriate changes and return a new node.

virtual void dynet::Nodeautobatch_reshape(const ComputationGraph &cg, const std::vector<VariableIndex> &batch_ids, const std::vector<int> &concat, std::vector<const Tensor *> &xs, Tensor &fx) const

reshape the tensors for auto Takes in info, and reshapes the dimensions of xs (for which “concat” is true), and fx. By default do no reshaping, which is OK for componentwise operations.

void dynet::Nodeautobatch_reshape_concatonly(const ComputationGraph &cg, const std::vector<VariableIndex> &batch_ids, const std::vector<int> &concat, std::vector<const Tensor *> &xs, Tensor &fx) const

reshape the tensors for auto Takes in info, and reshapes the dimensions of xs (for which “concat” is true) and fx by concatenating their batches.

unsigned dynet::Nodearity() const

Number of arguments to the function.

Return
Arity of the function

Public Members

std::vector<VariableIndex> dynet::Nodeargs

Dependency structure

Dim dynet::Nodedim

Will be .size() = 0 initially filled in by forward() TODO fix this

void *dynet::Nodeaux_mem

this will usually be null. but, if your node needs to store intermediate values between forward and backward, you can use store it here. request the number of bytes you need from aux_storage_size(). Note: this memory will be on the CPU or GPU, depending on your computation backend

Parameters and Model

Parameters are things that are optimized. in contrast to a system like Torch where computational modules may have their own parameters, in DyNet parameters are just parameters.

To deal with sparse updates, there are two parameter classes:

  • Parameters represents a vector, matrix, (eventually higher order tensors) of parameters. These are densely updated.
  • LookupParameters represents a table of vectors that are used to embed a set of discrete objects. These are sparsely updated.
struct dynet::ParameterStorageBase
#include <model.h>

This is the base class for ParameterStorage and LookupParameterStorage, the objects handling the actual parameters.

You can access the storage from any Parameter (resp. LookupParameter) class, use it only to do low level manipulations.

Subclassed by dynet::LookupParameterStorage, dynet::ParameterStorage

Public Functions

virtual void dynet::ParameterStorageBasescale_parameters(float a) = 0

Scale the parameters.

Parameters
  • a: scale factor

virtual void dynet::ParameterStorageBasescale_gradient(float a) = 0

Scale the gradient.

Parameters
  • a: scale factor

virtual void dynet::ParameterStorageBasezero() = 0

Set the parameters to 0.

virtual void dynet::ParameterStorageBasesquared_l2norm(float *sqnorm) const = 0

Get the parameter squared l2 norm.

Parameters
  • sqnorm: Pointer to the float holding the result

virtual void dynet::ParameterStorageBaseg_squared_l2norm(float *sqnorm) const = 0

Get the squared l2 norm of the gradient w.r.t. these parameters.

Parameters
  • sqnorm: Pointer to the float holding the result

virtual bool dynet::ParameterStorageBaseis_updated() const = 0

Check whether corpus is updated.

virtual bool dynet::ParameterStorageBasehas_grad() const = 0

Check whether the gradient is zero or not (true if gradient is non-zero)

virtual size_t dynet::ParameterStorageBasesize() const = 0

Get the size (number of scalar parameters)

Return
Number of scalar parameters

struct dynet::ParameterStorage
#include <model.h>

Storage class for Parameters.

Inherits from dynet::ParameterStorageBase

Subclassed by dynet::ParameterStorageCreator

Public Functions

void dynet::ParameterStoragecopy(const ParameterStorage &val)

Copy from another ParameterStorage.

Parameters

void dynet::ParameterStorageaccumulate_grad(const Tensor &g)

Add a tensor to the gradient.

After this method gets called, g <- g + d

Parameters

void dynet::ParameterStorageclear()

Clear the gradient (set it to 0)

void dynet::ParameterStorageclip(float left, float right)

Clip the values to the range [left, right].

Public Members

std::string dynet::ParameterStoragename

Name of this parameter

Dim dynet::ParameterStoragedim

Dimensions of the parameter tensor

Tensor dynet::ParameterStoragevalues

Values of the parameter

Tensor dynet::ParameterStorageg

Values of the gradient w.r.t. this parameter

bool dynet::ParameterStorageupdated

Whether this is updated

bool dynet::ParameterStoragenonzero_grad

Whether the gradient is zero

ParameterCollection *dynet::ParameterStorageowner

Pointer to the collection that “owns” this parameter

struct dynet::LookupParameterStorage
#include <model.h>

Storage class for LookupParameters.

Inherits from dynet::ParameterStorageBase

Subclassed by dynet::LookupParameterStorageCreator

Public Functions

void dynet::LookupParameterStorageinitialize(unsigned index, const std::vector<float> &val)

Initialize one particular lookup.

Parameters
  • index: Index of the lookput to initialize
  • val: Values

void dynet::LookupParameterStoragecopy(const LookupParameterStorage &val)

Copy from another LookupParameterStorage.

Parameters

void dynet::LookupParameterStorageaccumulate_grad(const Tensor &g)

Add a Tensor to the gradient of the whole lookup matrix.

after this grads<-grads + g

Parameters
  • g: [description]

void dynet::LookupParameterStorageaccumulate_grad(unsigned index, const Tensor &g)

Add a Tensor to the gradient of one of the lookups.

after this grads[index]<-grads[index] + g

Parameters
  • index: [description]
  • g: [description]

void dynet::LookupParameterStorageaccumulate_grads(unsigned n, const unsigned *ids_host, const unsigned *ids_dev, float *g)

Add tensors to muliple lookups.

After this method gets called, grads[ids_host[i]] <- grads[ids_host[i]] + g[i*dim.size():(i+1)*dim.size()]

Parameters
  • n: size of ids_host
  • ids_host: Indices of the gradients to update
  • ids_dev: [To be documented] (only for GPU)
  • g: Values

Public Members

std::string dynet::LookupParameterStoragename

Name of this parameter

Dim dynet::LookupParameterStorageall_dim

Total dimension

Tensor dynet::LookupParameterStorageall_values

Values for all dimensions at once

Tensor dynet::LookupParameterStorageall_grads

Gradient values for all dimensions at once

Dim dynet::LookupParameterStoragedim

Dimension for one lookup

std::vector<Tensor> dynet::LookupParameterStoragevalues

List of values for each lookup

std::vector<Tensor> dynet::LookupParameterStoragegrads

List of gradient values for each lookup

std::unordered_set<unsigned> dynet::LookupParameterStoragenon_zero_grads

Gradients are sparse, so track which components are nonzero

bool dynet::LookupParameterStorageupdated

Whether this lookup parameter should be updated

bool dynet::LookupParameterStoragenonzero_grad

Whether all of the gradients have been updated. Whether the gradient is zero

ParameterCollection *dynet::LookupParameterStorageowner

Pointer to the collection that “owns” this parameter

struct dynet::Parameter
#include <model.h>

Object representing a trainable parameter.

This objects acts as a high level component linking the actual parameter values (ParameterStorage) and the ParameterCollection. As long as you don’t want to do low level hacks at the ParameterStorage level, this is what you will use.

Public Functions

dynet::ParameterParameter()

Default constructor.

dynet::ParameterParameter(std::shared_ptr<ParameterStorage> p)

Constructor.

This is called by the model, you shouldn’t need to use it

Parameters
  • p: Shared pointer to the parameter storage

ParameterStorage &dynet::Parameterget_storage() const

Get underlying ParameterStorage object.

Return
ParameterStorage holding the parameter values

string dynet::Parameterget_fullname() const

Get the full name of the ParameterStorage object.

void dynet::Parameterzero()

Zero the parameters.

Dim dynet::Parameterdim() const

Shape of the parameter.

Return
Shape as a Dim object

Tensor *dynet::Parametervalues()

Values of the parameter.

Return
Values as a Tensor object

Tensor *dynet::Parametergradients()

gradients of the parameter

Return
gradients as a Tensor object

float dynet::Parametercurrent_weight_decay() const

Get the current weight decay for the parameters.

void dynet::Parameterset_updated(bool b)

Set the parameter as updated.

Parameters
  • b: Update status

void dynet::Parameterscale(float s)

Scales the parameter (multiplies by s)

Parameters
  • s: scale

void dynet::Parameterscale_gradient(float s)

Scales the gradient (multiplies by s)

Parameters
  • s: scale

bool dynet::Parameteris_updated()

Check the update status.

Return
Update status

void dynet::Parameterclip_inplace(float left, float right)

Clip the values of the parameter to the range [left, right] (in place)

void dynet::Parameterset_value(const std::vector<float> &val)

set the values of the parameter

Public Members

std::shared_ptr<ParameterStorage> dynet::Parameterp

Pointer to the storage for this Parameter

struct dynet::LookupParameter
#include <model.h>

Object representing a trainable lookup parameter.

Public Functions

LookupParameterStorage &dynet::LookupParameterget_storage() const

Get underlying LookupParameterStorage object.

Return
LookupParameterStorage holding the parameter values

void dynet::LookupParameterinitialize(unsigned index, const std::vector<float> &val) const

Initialize one particular column.

Parameters
  • index: Index of the column to be initialized
  • val: [description]

void dynet::LookupParameterzero()

Zero the parameters.

string dynet::LookupParameterget_fullname() const

Get the full name of the ParameterStorage object.

Dim dynet::LookupParameterdim() const

Shape of the lookup parameter.

Return
Shape as a Dim object

std::vector<Tensor> *dynet::LookupParametervalues()

Values of the lookup parameter.

Return
Values as a Tensor object

float dynet::LookupParametercurrent_weight_decay() const

Get the current weight decay for the parameters.

void dynet::LookupParameterscale(float s)

Scales the parameter (multiplies by s)

Parameters
  • s: scale

void dynet::LookupParameterscale_gradient(float s)

Scales the gradient (multiplies by s)

Parameters
  • s: scale

void dynet::LookupParameterset_updated(bool b)

Set the parameter as updated.

Parameters
  • b: Update status

bool dynet::LookupParameteris_updated()

Check the update status.

Return
Update status

Public Members

std::shared_ptr<LookupParameterStorage> dynet::LookupParameterp

Pointer to the storage for this Parameter

class dynet::ParameterCollection
#include <model.h>

This is a collection of parameters.

if you need a matrix of parameters, or a lookup table - ask an instance of this class. This knows how to serialize itself. Parameters know how to track their gradients, but any extra information (like velocity) will live here

Subclassed by dynet::Model

Public Functions

dynet::ParameterCollectionParameterCollection()

Constructor.

float dynet::ParameterCollectiongradient_l2_norm() const

Returns the l2 of your gradient.

Use this to look for gradient vanishing/exploding

Return
L2 norm of the gradient

void dynet::ParameterCollectionreset_gradient()

Sets all gradients to zero.

Parameter dynet::ParameterCollectionadd_parameters(const Dim &d, float scale = 0.0f, const std::string &name = "", Device *device = dynet::default_device)

Add parameters to model and returns Parameter object.

creates a ParameterStorage object holding a tensor of dimension d and returns a Parameter object (to be used as input in the computation graph). The coefficients are sampled according to the scale parameter

Return
Parameter object to be used in the computation graph
Parameters
  • d: Shape of the parameter
  • scale: If scale is non-zero, initializes according to \(mathcal U([-\mathrm{scale},+\mathrm{scale}]\), otherwise uses Glorot initialization
  • name: Name of the parameter
  • device: Device placement for the parameter

Parameter dynet::ParameterCollectionadd_parameters(const Dim &d, Device *device)

Add parameters to model and returns Parameter object.

creates a ParameterStorage object holding a tensor of dimension d and returns a Parameter object (to be used as input in the computation graph).

Return
Parameter object to be used in the computation graph
Parameters
  • d: Shape of the parameter
  • device: Device placement for the parameter

Parameter dynet::ParameterCollectionadd_parameters(const Dim &d, const std::string &name, Device *device = dynet::default_device)

Add parameters to model and returns Parameter object.

creates a ParameterStorage object holding a tensor of dimension d and returns a Parameter object (to be used as input in the computation graph).

Return
Parameter object to be used in the computation graph
Parameters
  • d: Shape of the parameter
  • name: Name of the parameter
  • device: Device placement for the parameter

Parameter dynet::ParameterCollectionadd_parameters(const Dim &d, const ParameterInit &init, const std::string &name = "", Device *device = dynet::default_device)

Add parameters with custom initializer.

Return
Parameter object to be used in the computation graph
Parameters
  • d: Shape of the parameter
  • init: Custom initializer
  • name: Name of the parameter
  • device: Device placement for the parameter

std::vector<std::shared_ptr<ParameterStorageBase>> dynet::ParameterCollectionget_parameter_storages_base() const

Get parameters base in current model.

Return
list of points to ParameterStorageBase objects

std::shared_ptr<ParameterStorage> dynet::ParameterCollectionget_parameter_storage(const std::string &pname)

Get parameter in current model.

It is not recommended to use this

Return
the pointer to the Parameter object

std::vector<std::shared_ptr<ParameterStorage>> dynet::ParameterCollectionget_parameter_storages() const

Get parameters in current model.

Return
list of points to ParameterStorage objects

LookupParameter dynet::ParameterCollectionadd_lookup_parameters(unsigned n, const Dim &d, const std::string &name = "", Device *device = dynet::default_device)

Add lookup parameter to model.

Same as add_parameters. Initializes with Glorot

Return
LookupParameter object to be used in the computation graph
Parameters
  • n: Number of lookup indices
  • d: Dimension of each embedding
  • name: Name of the parameter
  • device: Device placement for the parameter

LookupParameter dynet::ParameterCollectionadd_lookup_parameters(unsigned n, const Dim &d, const ParameterInit &init, const std::string &name = "", Device *device = dynet::default_device)

Add lookup parameter with custom initializer.

Return
LookupParameter object to be used in the computation graph
Parameters
  • n: Number of lookup indices
  • d: Dimension of each embedding
  • init: Custom initializer
  • name: Name of the parameter
  • device: Device placement for the parameter

std::shared_ptr<LookupParameterStorage> dynet::ParameterCollectionget_lookup_parameter_storage(const std::string &lookup_pname)

Get lookup parameter in current model.

It is not recommended to use this

Return
the pointer to the LookupParameter object

std::vector<std::shared_ptr<LookupParameterStorage>> dynet::ParameterCollectionget_lookup_parameter_storages() const

Get lookup parameters in current model.

Return
list of points to LookupParameterStorage objects

void dynet::ParameterCollectionproject_weights(float radius = 1.0f)

project weights so their L2 norm = radius

NOTE (Paul) : I am not sure this is doing anything currently. The argument doesn’t seem to be used anywhere… If you need this raise an issue on github

Parameters
  • radius: Target norm

void dynet::ParameterCollectionset_weight_decay_lambda(float lambda)

Set the weight decay coefficient.

Parameters
  • lambda: Weight decay coefficient

const std::vector<std::shared_ptr<ParameterStorage>> &dynet::ParameterCollectionparameters_list() const

Returns list of shared pointers to ParameterSorages.

You shouldn’t need to use this

Return
List of shared pointers to ParameterSorages

const std::vector<std::shared_ptr<LookupParameterStorage>> &dynet::ParameterCollectionlookup_parameters_list() const

Returns list of pointers to LookupParameterSorages.

You shouldn’t need to use this

Return
List of pointers to LookupParameterSorages

size_t dynet::ParameterCollectionparameter_count() const

Returns the total number of tunable parameters (i. e. scalars) contained within this model.

That is to say, a 2x2 matrix counts as four parameters.

Return
Number of parameters

size_t dynet::ParameterCollectionupdated_parameter_count() const

Returns total number of (scalar) parameters updated.

Return
number of updated parameters

void dynet::ParameterCollectionset_updated_param(const Parameter *p, bool status)

[brief description]

[long description]

Parameters
  • p: [description]
  • status: [description]

void dynet::ParameterCollectionset_updated_lookup_param(const LookupParameter *p, bool status)

[brief description]

[long description]

Parameters
  • p: [description]
  • status: [description]

bool dynet::ParameterCollectionis_updated_param(const Parameter *p)

[brief description]

[long description]

Return
[description]
Parameters
  • p: [description]

bool dynet::ParameterCollectionis_updated_lookup_param(const LookupParameter *p)

[brief description]

[long description]

Return
[description]
Parameters
  • p: [description]

ParameterCollection dynet::ParameterCollectionadd_subcollection(const std::string &name = "")

Add a sub-collection.

This will allow you to add a ParameterCollection that is a (possibly named) subset of the original collection. This is useful if you want to save/load/update only part of the parameters in the model.

Return
The subcollection

size_t dynet::ParameterCollectionsize()

Get size.

Get the number of parameters in the ParameterCollection

std::string dynet::ParameterCollectionget_fullname() const

get namespace of current ParameterCollection object(end with a slash)

L2WeightDecay &dynet::ParameterCollectionget_weight_decay()

Get the weight decay object.

struct dynet::ParameterInit
#include <param-init.h>

Initializers for parameters.

Allows for custom parameter initialization

Subclassed by dynet::ParameterInitConst, dynet::ParameterInitFromFile, dynet::ParameterInitFromVector, dynet::ParameterInitGlorot, dynet::ParameterInitIdentity, dynet::ParameterInitNormal, dynet::ParameterInitSaxe, dynet::ParameterInitUniform

Public Functions

dynet::ParameterInitParameterInit()

Default constructor.

virtual void dynet::ParameterInitinitialize_params(Tensor &values) const = 0

Function called upon initialization.

Whenever you inherit this struct to implement your own custom initializer, this is the function you want to overload to implement your logic.

Parameters
  • values: The tensor to be initialized. You should modify it in-place. See dynet/model.cc for some examples

struct dynet::ParameterInitNormal
#include <param-init.h>

Initialize parameters with samples from a normal distribution.

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitNormalParameterInitNormal(float m = 0.0f, float v = 1.0f)

Constructor.

Parameters
  • m: Mean of the gaussian distribution
  • v: Variance of the gaussian distribution (reminder : the variance is the square of the standard deviation)

struct dynet::ParameterInitUniform
#include <param-init.h>

Initialize parameters with samples from a uniform distribution.

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitUniformParameterInitUniform(float scale)

Constructor for uniform distribution centered on 0.

[long description]Samples parameters from \(mathcal U([-\mathrm{scale},+\mathrm{scale}]\)

Parameters
  • scale: Scale of the distribution

dynet::ParameterInitUniformParameterInitUniform(float l, float r)

Constructor for uniform distribution in a specific interval.

[long description]

Parameters
  • l: Lower bound of the interval
  • r: Upper bound of the interval

struct dynet::ParameterInitConst
#include <param-init.h>

Initialize parameters with a constant value.

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitConstParameterInitConst(float c)

Constructor.

Parameters
  • c: Constant value

struct dynet::ParameterInitIdentity
#include <param-init.h>

Initialize as the identity.

This will raise an exception if used on non square matrices

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitIdentityParameterInitIdentity()

Constructor.

struct dynet::ParameterInitGlorot
#include <param-init.h>

Initialize with the methods described in Glorot, 2010

In order to preserve the variance of the forward and backward flow across layers, the parameters \(\theta\) are initialized such that \(\mathrm{Var}(\theta)=\frac 2 {n_1+n_2}\) where \(n_1,n_2\) are the input and output dim. Important note : The underlying distribution is uniform (not gaussian)

Note: This is also known as Xavier initialization

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitGlorotParameterInitGlorot(bool is_lookup = false, float gain = 1.f)

Constructor.

Parameters
  • is_lookup: Boolean value identifying the parameter as a LookupParameter
  • gain: Scaling parameter. In order for the Glorot initialization to be correct, you should ût this equal to \(\frac 1 {f'(0)}\) where \(f\) is your activation function

struct dynet::ParameterInitSaxe
#include <param-init.h>

Initializes according to Saxe et al., 2014

Initializes as a random orthogonal matrix (unimplemented for GPU)

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitSaxeParameterInitSaxe(float gain = 1.0)

Constructor.

struct dynet::ParameterInitFromFile
#include <param-init.h>

Initializes from a file.

Useful for reusing weights, etc…

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitFromFileParameterInitFromFile(std::string f)

Constructor.

Parameters
  • f: File name (format should just be a list of values)

struct dynet::ParameterInitFromVector
#include <param-init.h>

Initializes from a std::vector of floats.

Inherits from dynet::ParameterInit

Public Functions

dynet::ParameterInitFromVectorParameterInitFromVector(std::vector<float> v)

Constructor.

Parameters
  • v: Vector of values to be used

Tensor

Tensor objects provide a bridge between C++ data structures and Eigen Tensors for multidimensional data.

Concretely, as an end user you will obtain a tensor object after calling .value() on an expression. You can then use functions described below to convert these tensors to float s, arrays of float s, to save and load the values, etc…

Conversely, when implementing low level nodes (e.g. for new operations), you will need to retrieve Eigen tensors from dynet tensors in order to perform efficient computation.

vector<Eigen::DenseIndex> dynet::as_vector(const IndexTensor &v)

Get the array of indices in an index tensor.

For higher order tensors this returns the flattened value

Return
Index values
Parameters
  • v: Input index tensor

std::ostream &dynet::operator<<(std::ostream &os, const Tensor &t)

You can use cout<<tensor; for debugging or saving.

Parameters

real dynet::as_scalar(const Tensor &t)

Get a scalar value from an order 0 tensor.

Throws an runtime_error exception if the tensor has more than one element.

TODO : Change for custom invalid dimension exception maybe?

Return
Scalar value
Parameters
  • t: Input tensor

std::vector<real> dynet::as_vector(const Tensor &v)

Get the array of values in the tensor.

For higher order tensors this returns the flattened value

Return
Values
Parameters
  • v: Input tensor

real dynet::rand01()

This is a helper function to sample uniformly in \([0,1]\).

Return
\(x\sim\mathcal U([0,1])\)

int dynet::rand0n(int n)

This is a helper function to sample uniformly in \(\{0,\dots,n-1\}\).

Return
\(x\sim\mathcal U(\{0,\dots,n-1\})\)
Parameters
  • n: Upper bound (excluded)

real dynet::rand_normal()

This is a helper function to sample from a normalized gaussian distribution.

Return
\(x\sim\mathcal N(0,1)\)

struct dynet::IndexTensor
#include <index-tensor.h>

Represents a tensor of indices.

This holds indices to locations within a dimension or tensor.

Public Functions

dynet::IndexTensorIndexTensor()

Create an empty tensor.

dynet::IndexTensorIndexTensor(const Dim &d, Eigen::DenseIndex *v, Device *dev, DeviceMempool mem)

Creates a tensor.

[long description]

Parameters
  • d: Shape of the tensor
  • v: Pointer to the values
  • dev: Device
  • mem: Memory pool

Public Members

Dim dynet::IndexTensord

Shape of tensor

Eigen::DenseIndex *dynet::IndexTensorv

Pointer to memory

struct dynet::Tensor
#include <tensor.h>

Represents a tensor of any order.

This provides a bridge between classic C++ types and Eigen tensors.

Public Functions

dynet::TensorTensor()

Create an empty tensor.

dynet::TensorTensor(const Dim &d, float *v, Device *dev, DeviceMempool mem)

Creates a tensor.

[long description]

Parameters
  • d: Shape of the tensor
  • v: Pointer to the values
  • dev: Device
  • mem: Memory pool

float *dynet::Tensorbatch_ptr(unsigned bid)

Get the pointer for a particular batch.

Automatically broadcasting if the size is zero

Return
Pointer to the memory where the batch values are located
Parameters
  • bid: Batch id requested

bool dynet::Tensoris_valid() const

Check for NaNs and infinite values.

This is very slow: use sparingly (it’s linear in the number of elements). This raises a std::runtime_error exception if the Tensor is on GPU because it’s not implemented yet

Return
Whether the tensor contains any invalid value

Tensor dynet::Tensorbatch_elem(unsigned b) const

Get a Tensor object representing a single batch.

If this tensor only has a single batch, then broadcast. Otherwise, check to make sure that the requested batch is smaller than the number of batches.

TODO: This is a bit wasteful, as it re-calculates bs.batch_size() every time.

Return
Sub tensor at batch b
Parameters
  • b: Batch id

std::vector<Tensor> dynet::Tensorbatch_elems() const

Get tensors for all batches.

Return
List of the tensors in each batch

Public Members

Dim dynet::Tensord

Shape of tensor

float *dynet::Tensorv

Pointer to memory

struct dynet::TensorTools
#include <tensor.h>

Provides tools for creating, accessing, copying and modifying tensors (in-place)

Public Static Functions

void dynet::TensorToolsclip(Tensor &d, float left, float right)

Clip the values in the tensor to a fixed range.

Parameters
  • d: Tensor to modify
  • left: Target minimum value
  • right: Target maximum value

void dynet::TensorToolsconstant(Tensor &d, float c)

Fills the tensor with a constant value.

Parameters
  • d: Tensor to modify
  • c: Target value

void dynet::TensorToolszero(Tensor &d)

Fills a tensor with zeros.

Parameters
  • d: Input tensor

void dynet::TensorToolsidentity(Tensor &val)

Set the (order 2) tensor as the identity matrix.

this throws a runtime_error exception if the tensor isn’t a square matrix

Parameters
  • val: Input tensor

void dynet::TensorToolsrandomize_bernoulli(Tensor &val, real p, real scale = 1.0f)

Fill the tensor with bernoulli random variables and scale them by scale.

Parameters
  • val: Input tensor
  • p: Parameter of the bernoulli distribution
  • scale: Scale of the random variables

void dynet::TensorToolsrandomize_normal(Tensor &val, real mean = 0.0f, real stddev = 1.0f)

Fill the tensor with gaussian random variables.

Parameters
  • val: Input tensor
  • mean: Mean
  • stddev: Standard deviation

void dynet::TensorToolsrandomize_uniform(Tensor &val, real left = 0.0f, real right = 1.0f)

Fill the tensor with uniform random variables.

Parameters
  • val: Input tensor
  • left: Left bound of the interval
  • right: Right bound of the interval

void dynet::TensorToolsrandomize_orthonormal(Tensor &val, real scale = 1.0f)

Takes a square matrix tensor and sets it as a random orthonormal matrix.

More specifically this samples a random matrix with RandomizeUniform and then performs SVD and returns the left orthonormal matrix in the decomposition, scaled by scale

Parameters
  • val: Input tensor
  • scale: Value to which the resulting orthonormal matrix will be scaled

float dynet::TensorToolsaccess_element(const Tensor &v, int index)

Access element of the tensor by index in the values array.

AccessElement and SetElement are very, very slow (potentially) - use appropriately

Return
v.v[index]
Parameters
  • v: Tensor
  • index: Index in the memory

float dynet::TensorToolsaccess_element(const Tensor &v, const Dim &index)

Access element of the tensor by indices in the various dimension.

This only works for matrix shaped tensors (+ batch dimension). AccessElement and SetElement are very, very slow (potentially) - use appropriately

Return
(*v)(index[0], index[1])
Parameters
  • v: Tensor
  • index: Indices in the tensor

void dynet::TensorToolsset_element(const Tensor &v, int index, float value)

Set element of the tensor by index in the values array.

AccessElement and SetElement are very, very slow (potentially) - use appropriately

Parameters
  • v: Tensor
  • index: Index in the memory
  • value: Desired value

void dynet::TensorToolscopy_element(const Tensor &l, int lindex, Tensor &r, int rindex)

Copy element from one tensor to another (by index in the values array)

Parameters
  • l: Source tensor
  • lindex: Source index
  • r: Target tensor
  • rindex: Target index

void dynet::TensorToolsset_elements(const Tensor &v, const std::vector<float> &vec)

Set the elements of a tensor with an array of values.

(This uses memcpy so be careful)

Parameters

void dynet::TensorToolscopy_elements(Tensor &v, const Tensor &v_src)

Copy one tensor into another.

Parameters
  • v: Target tensor
  • v_src: Source tensor

void dynet::TensorToolsaccumulate(Tensor &v, const Tensor &v_src)

Accumulate the values of one tensor into another.

Parameters
  • v: Target tensor
  • v_src: Source tensor

void dynet::TensorToolslogsumexp(const Tensor &x, Tensor &m, Tensor &z, unsigned d = 0)

Calculate the logsumexp function over all columns of the tensor.

Parameters
  • x: The input tensor
  • m: A tensor of scratch memory to hold the maximum values of each column
  • z: The output tensor

IndexTensor dynet::TensorToolsargmax(const Tensor &v, unsigned dim = 0, unsigned num = 1)

Calculate the index of the maximum value.

Return
A newly allocated LongTensor consisting of argmax IDs. The length of the dimension “dim” will be “num”, consisting of the appropriate IDs.
Parameters
  • v: A tensor where each row represents a probability distribution
  • dim: Which dimension to take the argmax over
  • num: The number of kmax values

IndexTensor dynet::TensorToolscategorical_sample_log_prob(const Tensor &v, unsigned dim = 0, unsigned num = 1)

Calculate samples from a log probability.

Return
A newly allocated LongTensor consisting of argmax IDs. The length of the dimension “dim” will be “num”, consisting of the appropriate IDs.
Parameters
  • v: A tensor where each row represents a log probability distribution
  • dim: Which dimension to take the sample over
  • num: The number of samples for each row

Dimensions

The Dim class holds information on the shape of a tensor. As explained in Unorthodox Design, in DyNet the dimensions are represented as the standard dimension + the batch dimension, which makes batched computation transparent.

DYNET_MAX_TENSOR_DIM

Maximum number of dimensions supported by dynet : 7

struct dynet::Dim
#include <dim.h>

The Dim struct stores information about the dimensionality of expressions.

Batch dimension is treated separately from standard dimension.

Public Functions

dynet::DimDim()

Default constructor.

dynet::DimDim(std::initializer_list<unsigned int> x)

Initialize from a list of dimensions.

The batch dimension is 1 in this case (non-batched expression)

Parameters
  • x: List of dimensions

dynet::DimDim(std::initializer_list<unsigned int> x, unsigned int b)

Initialize from a list of dimensions and a batch size.

Parameters
  • x: List of dimensions
  • b: Batch size

dynet::DimDim(const std::vector<long> &x)

Initialize from a vector of dimensions.

The batch dimension is 1 in this case (non-batched expression)

Parameters
  • x: Array of dimensions

dynet::DimDim(const std::vector<long> &x, unsigned int b)

Initialize from a vector of dimensions and a batch size.

Parameters
  • x: Vector of dimensions
  • b: Batch size

unsigned int dynet::Dimsize() const

Total size of a batch.

Return
Batch size * size of a batch

unsigned int dynet::Dimbatch_size() const

Size of a batch (product of all dimensions)

Return
Size of a batch

unsigned int dynet::Dimsum_dims() const

Sum of all dimensions within a batch.

Return
Sum of the dimensions within a batch

Dim dynet::Dimtruncate() const

remove trailing dimensions of 1

iterate all the dimensions of Dim, stop at last dimension of 1

Return
truncated dimension

Dim dynet::Dimsingle_batch() const

Set the batch dimension to 1.

Return
1-batch version of this instance

void dynet::Dimresize(unsigned int i)

Change the number of dimensions.

Parameters
  • int: New number of dimensions

unsigned int dynet::Dimndims() const

Get number of dimensions.

Return
Number of dimensions

unsigned int dynet::Dimrows() const

Size of the first dimension.

Return
Size of the first dimension

unsigned int dynet::Dimnum_nonone_dims() const

Number of non-one dimensions.

Return
Number of non-one dimensions

unsigned int dynet::Dimcols() const

Size of the second dimension (or 1 if only one dimension)

Return
Size of the second dimension (or 1 if only one dimension)

unsigned int dynet::Dimbatch_elems() const

Batch dimension.

Return
Batch dimension

void dynet::Dimset(unsigned int i, unsigned int s)

Set specific dimension.

Set the value of a specific dimension to an arbitrary value

Parameters
  • i: Dimension index
  • s: Dimension size

unsigned int dynet::Dimoperator[](unsigned int i) const

Access a specific dimension as you would access an array element.

Return
Size of dimension i
Parameters
  • i: Dimension index

unsigned int dynet::Dimsize(unsigned int i) const

Size of dimension i.

Return
Size of dimension i
Parameters
  • i: Dimension index

void dynet::Dimdelete_dim(unsigned int i)

Remove one of the dimensions.

Parameters
  • i: index of the dimension to be removed

void dynet::Dimdelete_dims(std::vector<unsigned int> dims, bool reduce_batch)

Remove multi-dimensions.

Parameters
  • dims: dimensions to be removed
  • reduce_batch: reduce the batch dimension or not

void dynet::Dimadd_dim(unsigned int n)

Insert a dimension to the end.

Parameters
  • n: the size of the new dimension

void dynet::Diminsert_dim(unsigned int i, unsigned int n)

Insert a dimension.

Parameters
  • i: the index to insert the new dimension
  • n: the size of the new dimension

Dim dynet::Dimtranspose() const

Transpose a vector or a matrix.

This raises an invalid_argument exception on tensors with more than 2 dimensions

Return
The transposed Dim structure

void dynet::Dimprint_profile(std::ostream &out) const

Print the unbatched profile as a string.

Public Members

unsigned int dynet::Dimd[DYNET_MAX_TENSOR_DIM]

Array of dimension

unsigned int dynet::Dimnd

Number of dimensions

unsigned int dynet::Dimbd

Batch dimension