jactorch.functional.kernel

Useful utilities for kernel-based attention mechanism.

Functions

cosine_distance(f_lookup, f)	Cosine distance kernel.
dot(f_lookup, f)	Dot product kernel, essentially a cosine distance kernel without normalization.
inverse_distance(f_lookup, f[, p, eps])	Inverse distance kernel.

Functions

cosine_distance(f_lookup, f)

Cosine distance kernel.

Parameters:

• f_lookup (FloatTensor) – features of the lookup keys

• f (FloatTensor) – features of the value keys

Returns:

the attention mask for each lookup keys.

Return type:

FloatTensor

dot(f_lookup, f)

Dot product kernel, essentially a cosine distance kernel without normalization.

Parameters:

• f_lookup (FloatTensor) – features of the lookup keys

• f (FloatTensor) – features of the value keys

Returns:

the attention mask for each lookup keys.

Return type:

FloatTensor

inverse_distance(f_lookup, f, p=2, eps=1e-8)

Inverse distance kernel.

Parameters:

• f_lookup (FloatTensor) – features of the lookup keys

• f (FloatTensor) – features of the value keys

Returns:

the attention mask for each lookup keys.

Return type:

FloatTensor