Physics:Min-entropy

The min-entropy, in information theory, is the smallest of the Rényi family of entropies, corresponding to the most conservative way of measuring the unpredictability of a set of outcomes, as the negative logarithm of the probability of the most likely outcome. The various Rényi entropies are all equal for a uniform distribution, but measure the unpredictability of a nonuniform distribution in different ways. The min-entropy is never greater than the ordinary or Shannon entropy (which measures the average unpredictability of the outcomes) and that in turn is never greater than the Hartley or max-entropy, defined as the logarithm of the number of outcomes with nonzero probability. As with the classical Shannon entropy and its quantum generalization, the von Neumann entropy, one can define a conditional version of min-entropy. The conditional quantum min-entropy is a one-shot, or conservative, analog of conditional quantum entropy.

To interpret a conditional information measure, suppose Alice and Bob were to share a bipartite quantum state [math]\displaystyle{ \rho_{AB} }[/math]. Alice has access to system [math]\displaystyle{ A }[/math] and Bob to system [math]\displaystyle{ B }[/math]. The conditional entropy measures the average uncertainty Bob has about Alice's state upon sampling from his own system. The min-entropy can be interpreted as the distance of a state from a maximally entangled state.

This concept is useful in quantum cryptography, in the context of privacy amplification (See for example ).

Definition for classical distributions

If [math]\displaystyle{ P=(p_1,...,p_n) }[/math] is a classical finite probability distribution, its min-entropy can be defined as [math]\displaystyle{ H_{\rm min}(\boldsymbol P) = \log\frac{1}{P_{\rm max}}, \qquad P_{\rm max}\equiv \max_i p_i. }[/math]One way to justify the name of the quantity is to compare it with the more standard definition of entropy, which reads [math]\displaystyle{ H(\boldsymbol P)=\sum_i p_i\log(1/p_i) }[/math], and can thus be written concisely as the expectation value of [math]\displaystyle{ \log (1/p_i) }[/math] over the distribution. If instead of taking the expectation value of this quantity we take its minimum value, we get precisely the above definition of [math]\displaystyle{ H_{\rm min}(\boldsymbol P) }[/math].

Definition for quantum states

A natural way to define a "min-entropy" for quantum states is to leverage the simple observation that quantum states result in probability distributions when measured in some basis. There is however the added difficulty that a single quantum state can result in infinitely many possible probability distributions, depending on how it is measured. A natural path is then, given a quantum state [math]\displaystyle{ \rho }[/math], to still define [math]\displaystyle{ H_{\rm min}(\rho) }[/math] as [math]\displaystyle{ \log(1/P_{\rm max}) }[/math], but this time defining [math]\displaystyle{ P_{\rm max} }[/math] as the maximum possible probability that can be obtained measuring [math]\displaystyle{ \rho }[/math], maximizing over all possible projective measurements.

Formally, this would provide the definition [math]\displaystyle{ H_{\rm min}(\rho) = \max_\Pi \log \frac{1}{\max_i \operatorname{tr}(\Pi_i \rho)} = - \max_\Pi \log \max_i \operatorname{tr}(\Pi_i \rho), }[/math]where we are maximizing over the set of all projective measurements [math]\displaystyle{ \Pi=(\Pi_i)_i }[/math], [math]\displaystyle{ \Pi_i }[/math] represent the measurement outcomes in the POVM formalism, and [math]\displaystyle{ \operatorname{tr}(\Pi_i \rho) }[/math] is therefore the probability of observing the [math]\displaystyle{ i }[/math]-th outcome when the measurement is [math]\displaystyle{ \Pi }[/math].

A more concise method to write the double maximization is to observe that any element of any POVM is a Hermitian operator such that [math]\displaystyle{ 0\le \Pi\le I }[/math], and thus we can equivalently directly maximize over these to get [math]\displaystyle{ H_{\rm min}(\rho) = - \max_{0\le \Pi\le I} \log \operatorname{tr}(\Pi \rho). }[/math]In fact, this maximization can be performed explicitly and the maximum is obtained when [math]\displaystyle{ \Pi }[/math] is the projection onto (any of) the largest eigenvalue(s) of [math]\displaystyle{ \rho }[/math]. We thus get yet another expression for the min-entropy as: [math]\displaystyle{ H_{\rm min}(\rho) = -\log \|\rho\|_{\rm op}, }[/math]remembering that the operator norm of a Hermitian positive semidefinite operator equals its largest eigenvalue.

Conditional entropies

Let [math]\displaystyle{ \rho_{AB} }[/math] be a bipartite density operator on the space [math]\displaystyle{ \mathcal{H}_A \otimes \mathcal{H}_B }[/math]. The min-entropy of [math]\displaystyle{ A }[/math] conditioned on [math]\displaystyle{ B }[/math] is defined to be

[math]\displaystyle{ H_{\min}(A|B)_{\rho} \equiv -\inf_{\sigma_B}D_{\max}(\rho_{AB}\|I_A \otimes \sigma_B) }[/math]

where the infimum ranges over all density operators [math]\displaystyle{ \sigma_B }[/math] on the space [math]\displaystyle{ \mathcal{H}_B }[/math]. The measure [math]\displaystyle{ D_{\max} }[/math] is the maximum relative entropy defined as

[math]\displaystyle{ D_{\max}(\rho\|\sigma) = \inf_{\lambda}\{\lambda:\rho \leq 2^{\lambda}\sigma\} }[/math]

The smooth min-entropy is defined in terms of the min-entropy.

[math]\displaystyle{ H_{\min}^{\epsilon}(A|B)_{\rho} = \sup_{\rho'} H_{\min}(A|B)_{\rho'} }[/math]

where the sup and inf range over density operators [math]\displaystyle{ \rho'_{AB} }[/math] which are [math]\displaystyle{ \epsilon }[/math]-close to [math]\displaystyle{ \rho_{AB} }[/math]. This measure of [math]\displaystyle{ \epsilon }[/math]-close is defined in terms of the purified distance

[math]\displaystyle{ P(\rho,\sigma) = \sqrt{1 - F(\rho,\sigma)^2} }[/math]

where [math]\displaystyle{ F(\rho,\sigma) }[/math] is the fidelity measure.

These quantities can be seen as generalizations of the von Neumann entropy. Indeed, the von Neumann entropy can be expressed as

[math]\displaystyle{ S(A|B)_{\rho} = \lim_{\epsilon\rightarrow 0}\lim_{n\rightarrow\infty}\frac{1}{n}H_{\min}^{\epsilon}(A^n|B^n)_{\rho^{\otimes n}}~. }[/math]

This is called the fully quantum asymptotic equipartition theorem. The smoothed entropies share many interesting properties with the von Neumann entropy. For example, the smooth min-entropy satisfy a data-processing inequality:

[math]\displaystyle{ H_{\min}^{\epsilon}(A|B)_{\rho} \geq H_{\min}^{\epsilon}(A|BC)_{\rho}~. }[/math]

Operational interpretation of smoothed min-entropy

Henceforth, we shall drop the subscript [math]\displaystyle{ \rho }[/math] from the min-entropy when it is obvious from the context on what state it is evaluated.

Min-entropy as uncertainty about classical information

Suppose an agent had access to a quantum system [math]\displaystyle{ B }[/math] whose state [math]\displaystyle{ \rho_{B}^x }[/math] depends on some classical variable [math]\displaystyle{ X }[/math]. Furthermore, suppose that each of its elements [math]\displaystyle{ x }[/math] is distributed according to some distribution [math]\displaystyle{ P_X(x) }[/math]. This can be described by the following state over the system [math]\displaystyle{ XB }[/math].

[math]\displaystyle{ \rho_{XB} = \sum_x P_X (x) |x\rangle\langle x| \otimes \rho_{B}^x , }[/math]

where [math]\displaystyle{ \{|x\rangle\} }[/math] form an orthonormal basis. We would like to know what the agent can learn about the classical variable [math]\displaystyle{ x }[/math]. Let [math]\displaystyle{ p_g(X|B) }[/math] be the probability that the agent guesses [math]\displaystyle{ X }[/math] when using an optimal measurement strategy

[math]\displaystyle{ p_g(X|B) = \sum_x P_X(x)tr(E_x \rho_B^x) , }[/math]

where [math]\displaystyle{ E_x }[/math] is the POVM that maximizes this expression. It can be shown that this optimum can be expressed in terms of the min-entropy as

[math]\displaystyle{ p_g(X|B) = 2^{-H_{\min}(X|B)}~. }[/math]

If the state [math]\displaystyle{ \rho_{XB} }[/math] is a product state i.e. [math]\displaystyle{ \rho_{XB} = \sigma_X \otimes \tau_B }[/math] for some density operators [math]\displaystyle{ \sigma_X }[/math] and [math]\displaystyle{ \tau_B }[/math], then there is no correlation between the systems [math]\displaystyle{ X }[/math] and [math]\displaystyle{ B }[/math]. In this case, it turns out that [math]\displaystyle{ 2^{-H_{\min}(X|B)} = \max_x P_X(x)~. }[/math]

Min-entropy as overlap with the maximally entangled state

The maximally entangled state [math]\displaystyle{ |\phi^+\rangle }[/math] on a bipartite system [math]\displaystyle{ \mathcal{H}_A \otimes \mathcal{H}_B }[/math] is defined as

[math]\displaystyle{ |\phi^+\rangle_{AB} = \frac{1}{\sqrt{d}} \sum_{x_A,x_B} |x_A\rangle |x_B\rangle }[/math]

where [math]\displaystyle{ \{|x_A\rangle\} }[/math] and [math]\displaystyle{ \{|x_B\rangle\} }[/math] form an orthonormal basis for the spaces [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] respectively. For a bipartite quantum state [math]\displaystyle{ \rho_{AB} }[/math], we define the maximum overlap with the maximally entangled state as

[math]\displaystyle{ q_{c}(A|B) = d_A \max_{\mathcal{E}} F\left((I_A \otimes \mathcal{E}) \rho_{AB}, |\phi^+\rangle\langle \phi^{+}|\right)^2 }[/math]

where the maximum is over all CPTP operations [math]\displaystyle{ \mathcal{E} }[/math] and [math]\displaystyle{ d_A }[/math] is the dimension of subsystem [math]\displaystyle{ A }[/math]. This is a measure of how correlated the state [math]\displaystyle{ \rho_{AB} }[/math] is. It can be shown that [math]\displaystyle{ q_c(A|B) = 2^{-H_{\min}(A|B)} }[/math]. If the information contained in [math]\displaystyle{ A }[/math] is classical, this reduces to the expression above for the guessing probability.

Proof of operational characterization of min-entropy

The proof is from a paper by König, Schaffner, Renner in 2008. It involves the machinery of semidefinite programs. Suppose we are given some bipartite density operator [math]\displaystyle{ \rho_{AB} }[/math]. From the definition of the min-entropy, we have

[math]\displaystyle{ H_{\min}(A|B) = - \inf_{\sigma_B} \inf_{\lambda} \{ \lambda | \rho_{AB} \leq 2^{\lambda}(I_A \otimes \sigma_B)\}~. }[/math]

This can be re-written as

[math]\displaystyle{ -\log \inf_{\sigma_B} \operatorname{Tr}(\sigma_B) }[/math]

subject to the conditions

[math]\displaystyle{ \sigma_B \geq 0 }[/math]

[math]\displaystyle{ I_A \otimes \sigma_B \geq \rho_{AB}~. }[/math]

We notice that the infimum is taken over compact sets and hence can be replaced by a minimum. This can then be expressed succinctly as a semidefinite program. Consider the primal problem

[math]\displaystyle{ \text{min:}\operatorname{Tr} (\sigma_B) }[/math]

[math]\displaystyle{ \text{subject to: } I_A \otimes \sigma_B \geq \rho_{AB} }[/math]

[math]\displaystyle{ \sigma_B \geq 0~. }[/math]

This primal problem can also be fully specified by the matrices [math]\displaystyle{ (\rho_{AB},I_B,\operatorname{Tr}^*) }[/math] where [math]\displaystyle{ \operatorname{Tr}^* }[/math] is the adjoint of the partial trace over [math]\displaystyle{ A }[/math]. The action of [math]\displaystyle{ \operatorname{Tr}^* }[/math] on operators on [math]\displaystyle{ B }[/math] can be written as

[math]\displaystyle{ \operatorname{Tr}^*(X) = I_A \otimes X~. }[/math]

We can express the dual problem as a maximization over operators [math]\displaystyle{ E_{AB} }[/math] on the space [math]\displaystyle{ AB }[/math] as

[math]\displaystyle{ \text{max:}\operatorname{Tr}(\rho_{AB}E_{AB}) }[/math]

[math]\displaystyle{ \text{subject to: } \operatorname{Tr}_A(E_{AB}) = I_B }[/math]

[math]\displaystyle{ E_{AB} \geq 0~. }[/math]

Using the Choi–Jamiołkowski isomorphism, we can define the channel [math]\displaystyle{ \mathcal{E} }[/math] such that

[math]\displaystyle{ d_A I_A \otimes \mathcal{E}^{\dagger}(|\phi^{+}\rangle\langle\phi^{+}|) = E_{AB} }[/math]

where the bell state is defined over the space [math]\displaystyle{ AA' }[/math]. This means that we can express the objective function of the dual problem as

[math]\displaystyle{ \langle \rho_{AB}, E_{AB} \rangle = d_A \langle \rho_{AB}, I_A \otimes \mathcal{E}^{\dagger} (|\phi^+\rangle\langle \phi^+|) \rangle }[/math]

[math]\displaystyle{ = d_A \langle I_A \otimes \mathcal{E}(\rho_{AB}), |\phi^+\rangle\langle \phi^+|) \rangle }[/math]

as desired.

Notice that in the event that the system [math]\displaystyle{ A }[/math] is a partly classical state as above, then the quantity that we are after reduces to

[math]\displaystyle{ \max P_X(x) \langle x | \mathcal{E}(\rho_B^x)|x \rangle~. }[/math]

We can interpret [math]\displaystyle{ \mathcal{E} }[/math] as a guessing strategy and this then reduces to the interpretation given above where an adversary wants to find the string [math]\displaystyle{ x }[/math] given access to quantum information via system [math]\displaystyle{ B }[/math].

See also

• von Neumann entropy
• Generalized relative entropy
• max-entropy