nzy/hw2

hw2/ZhongyiNi/hw924.md

@@ -0,0 +1,53 @@
+## 9.24
+Consider the perturbation of $ X $ by $ \epsilon Y $, where $ \epsilon $ is a small scalar. The potential at $ X + \epsilon Y $ is:
+$$
+V(X + \epsilon Y) = -\ln \det(X + \epsilon Y).
+$$
+
+To first order in $ \epsilon $, the determinant of $ X + \epsilon Y $ can be expanded as:
+$$
+\det(X + \epsilon Y) = \det(X) \det(I + \epsilon X^{-1} Y).
+$$
+Using the identity $ \det(I + \epsilon A) \approx 1 + \epsilon \text{tr}(A) $ for small $ \epsilon $, we get:
+$$
+\det(X + \epsilon Y) \approx \det(X) \left(1 + \epsilon \text{tr}(X^{-1} Y)\right).
+$$
+
+Taking the natural logarithm:
+$$
+\ln \det(X + \epsilon Y) \approx \ln \det X + \ln \left(1 + \epsilon \text{tr}(X^{-1} Y)\right) \approx \ln \det X + \epsilon \text{tr}(X^{-1} Y),
+$$
+where we used $ \ln(1 + a) \approx a $ for small $ a $.
+
+Thus, the potential $ V(X + \epsilon Y) $ is:
+$$
+V(X + \epsilon Y) \approx -\ln \det X - \epsilon \text{tr}(X^{-1} Y).
+$$
+
+The directional derivative of $ V $ at $ X $ in the direction $ Y $ is given by:
+$$
+V(X + \epsilon Y) = V(X) + \epsilon \langle \nabla V, Y \rangle + O(\epsilon^2).
+$$
+
+Comparing the two expressions, we identify:
+$$
+\langle \nabla V, Y \rangle = -\text{tr}(X^{-1} Y).
+$$
+
+The inner product $ \langle \nabla V, Y \rangle $ for matrices is typically the Frobenius inner product:
+$$
+\langle A, B \rangle = \text{tr}(A^T B).
+$$
+Thus,
+$$
+\langle \nabla V, Y \rangle = \text{tr}((\nabla V)^T Y) = -\text{tr}(X^{-1} Y).
+$$
+
+Since this holds for arbitrary $ Y $, we conclude:
+$$
+(\nabla V)^T = -X^{-1},
+$$
+or equivalently:
+$$
+\nabla V = -X^{-T}.
+$$

hw2/ZhongyiNi/hw927.md

@@ -0,0 +1,35 @@
+## 9.27
+
+### Integer Solutions Form a Lattice
+For a fixed $A x_p = b $, the set of all integer solutions is:
+$$ \{ x_p + x_h \mid Ax_h = 0, x_h \in \mathbb{Z}^d \}. $$
+
+The set $ \{ x_h \mid Ax_h = 0, x_h \in \mathbb{Z}^d \} $ is the set of integer solutions to the homogeneous system $ Ax = 0 $. This is a sublattice of $ \mathbb{Z}^d $ (a lattice contained within $ \mathbb{Z}^d $) because:
+1. It is closed under addition: if $ x_h, x_h' $ satisfy $ Ax_h = 0 $ and $ Ax_h' = 0 $, then $ A(x_h + x_h') = 0 $.
+2. It is closed under integer scaling: if $ Ax_h = 0 $ and $ k \in \mathbb{Z} $, then $ A(k x_h) = 0 $.
+3. It is discrete because $ \mathbb{Z}^d $ is discrete.
+
+Thus, the set of integer solutions to $ Ax = b $ is a translate (by $ x_p $) of a sublattice of $ \mathbb{Z}^d $, which is itself a lattice.
+
+
+We need to find one integer solution $ x_p $ to $ Ax = b $. This can be done using the SVD of $ A $
+$$ A = U S V $$
+
+The system $ Ax = b $ becomes:
+$$ U S V x = b \implies S (V x) = U^{-1} b. $$
+Let $ y = V x $ and $ b' = U^{-1} b $. Then the system is:
+$$ S y = b'. $$
+
+This system is easy to solve because $ S $ is diagonal.
+
+Given that the set of integer solutions is a lattice $ \{ x_p + \sum_{i=1}^{d-r} k_i v_i \mid k_i \in \mathbb{Z} \} $, the optimization problem becomes:
+$$ \max c^T (x_p + \sum_{i=1}^{d-r} k_i v_i). $$
+
+This is equivalent to:
+$$ \max \sum_{i=1}^{d-r} (c^T v_i) k_i + c^T x_p. $$
+
+Since $ k_i \in \mathbb{Z} $, this is an unbounded problem unless $ c^T v_i = 0 $ for all $ i $. If $ c^T v_i \neq 0 $ for some $ i $, we can make $ c^T x $ arbitrarily large by choosing $ k_i $ appropriately (positive or negative depending on the sign of $ c^T v_i $). 
+
+Thus:
+1. If $ c^T v_i = 0 $ for all $ i $, then $ c^T x = c^T x_p $ is constant over all solutions, and $ x_p $ is optimal.
+2. Otherwise, the problem is unbounded.