diag(x)

When using \({\bf diag}(x)\) in a text, there is always the nagging feeling that there must be a nice way to express this in standard matrix algebra (i.e., some combination of identity matrices, all-ones vectors, and standard matrix multiplications). To remind ourselves, the \({\bf diag}(x)\) function creates a diagonal matrix with \(x_i\) as diagonal elements: \[{\bf diag}(x) = \begin{bmatrix} x_1 & & & \\ & x_2 & & \\ & & \ddots & \\ & & & x_n \end{bmatrix} \] Obviously, my intuition is wrong here, as I have never seen such formulation, and \({\bf diag}(x)\) is used all over the place. I have seen some attempts, but they require exotic notation, and, as a result, don't really improve upon straight use of \({\bf diag}(x)\).

Here are some examples from [1].

Hadamard (elementwise) product

The Hadamard product, usually written with the \(\circ\) operator (although \(\odot\) is also used), defines an elementwise product between two equally sized matrices: \[ C = A \circ B \iff c_{i,j} = a_{i,j} \cdot b_{i,j}\] With this, we can form: \[ {\bf diag}(x) = (x{\bf 1}^T)\circ I \] Here \({\bf 1}\) is a column-vector of ones. The first operation constructs a matrix with \(n\) identical columns \(x\). After that, the Hadamard product with the identity matrix will erase all off-diagonal elements. 

Kronecker product

Another funky operator is the Kronecker product, denoted by \(\otimes\). This is defined as: \[ A \otimes B = \begin{bmatrix} a_{1,1} B & a_{1,2} B & \cdots & a_{1,n} B\\ a_{2,1} B & a_{2,2} B & \cdots & a_{2,n} B \\ \vdots & \vdots & \ddots & \vdots \\ a_{n,1}B & a_{n,2}B & \cdots & a_{n,n}B \end{bmatrix}\] So, if \(A\) is an \((m\times n)\) matrix, and \(B\) is \((p\times q)\), then \(A \otimes B\) is \((mp \times nq)\). This is not so useful for us, but I noticed that someone wrote \(x{\bf 1}^T\) as \[x{\bf 1}^T = {\bf 1}^T \otimes x\] That is not much of a simplification (and you still need the Hadamar product to form \({\bf diag}(x)\)).

Tensors

Tensors are a multi-dimensional generalization of the 2-dimensional matrix. The 3d identity tensor is \[I_{i,j,k} = \begin{cases}1 & \text{if $i=j=k$} \\ 0 & \text{otherwise}\end{cases}\] We can use the tensor product to form: \[{\bf diag}(x) = I x\] This is the simplest, but we need to go beyond standard matrix algebra here. 

Test using pytorch:

Inverse

Actually, I needed the inverse of the diagonal matrix. So that means we need to choose between: \[ {\bf diag}(x)^{-1} \] and \[ {\bf diag}^{-1}(x) \] To work around this dilemma, it may be better to introduce \[X =  {\bf diag}(x) \] so that we can write \(X^{-1}\). 

Conclusion

I still have not found a nice expression in matrix algebra that forms \({\bf diag}(x)\). In all likelihood, it does not exist.

References

1. Convert vector into diagonal matrix, https://math.stackexchange.com/questions/1731183/convert-vector-into-diagonal-matrix/2540232