# 注释 (2023/4/2 上午11:56:04) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=222&annotation=5LM7JWAU)“The term blind indicates that the source signals can be separated even if little information is known about the source signals” ([Tharwat, 2021, p. 222](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=223&annotation=A7C3ZZU8)“PCA finds uncorrelated components while ICA finds independent components” ([Tharwat, 2021, p. 223](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=223&annotation=E6VL85YW)“The main goal of these algorithms is to extract independent components by (1) maximizing the non-Gaussianity, (2) minimizing the mutual information, or (3) using maximum likelihood (ML) estimation method” ([Tharwat, 2021, p. 223](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=226&annotation=LE5KXVFF)“the source signals must be non-Gaussian, and this assumption is a fundamental restriction in ICA” ([Tharwat, 2021, p. 226](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=226&annotation=3JY627XI)“the ICA model cannot estimate Gaussian independent components” ([Tharwat, 2021, p. 226](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=226&annotation=RPYC4KB2)“if the extracted signals from mixture signals are independent, have non-Gaussian histograms, or have low complexity than mixture signals; then these signals represent source signals.” ([Tharwat, 2021, p. 226](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=229&annotation=HLQ6253G)“the length of the weight vector affects only the amplitude of the extracted signal.” ([Tharwat, 2021, p. 229](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=231&annotation=XSL6RRP7)“Ambiguities of ICA” ([Tharwat, 2021, p. 231](zotero://select/library/items/NAHTFCR3)) 1.不能确定独立成分的方差或者说能量 因为A和s都是未知的，s缩放的倍数可以在A上补偿回来。 因此我们统一假设各独立分量的方差为1，。其实，这样假设之后仍然存在一个不确定性，独立分量乘以-1并不影响方差。不过在实际应用时，并不影响。 2.不能确定各独立分量的顺序 同样，由于A和s都是未知的，在预测时我们可以任意改变A和s对应分量的位置，在矩阵运算中就是乘以一个置换矩阵。 引用自：https://blog.csdn.net/u014485485/article/details/78452820 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=231&annotation=M72H4VVN)“The order of independent components” ([Tharwat, 2021, p. 231](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=231&annotation=2USQ8C6K)“The sign of independent components:” ([Tharwat, 2021, p. 231](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=231&annotation=NE8P3G7A)“This phase has two main steps: centering and whitening.” ([Tharwat, 2021, p. 231](zotero://select/library/items/NAHTFCR3)) ICA的预处理包括两个步骤： 1. 中心化 2. 白化 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=231&annotation=NIUMR3LX)“The mean vector can be added back to independent components after applying ICA” ([Tharwat, 2021, p. 231](zotero://select/library/items/NAHTFCR3)) 在应用ICA算法后，均值矩阵可以加回到独立分量中 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=232&annotation=UE62GXD2)“In ICA, the PCA technique can be used for decorrelating signals.” ([Tharwat, 2021, p. 232](zotero://select/library/items/NAHTFCR3)) ICA中可以利用PCA进行信号的去相关处理 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=45Y7D259)“The first is based on the non-Gaussianity. This can be measured by some measures such as negentropy and kurtosis, and the goal of this approach is to find independent components which maximize the non-Gaussianity” ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=HPZ7HT8Y)“n the second approach, the ICA goal can be obtained by minimizing the mutual information” ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=MKEGTZIB)“Independent components can be also estimated by using maximum likelihood (ML) estimation” ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) ICA有三种评估分量独立性的方法： 1. 基于非高斯度 2. 最小化互信息度 3. 基于最大似然估计 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=3K5BGNB5)“All approaches simply search for a rotation or unmixing matrix W. Projecting the whitened data onto that rotation matrix extracts independent signals.” ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) 这些方法的本质都是寻找一个旋转矩阵，并将白化后的数据映射到旋转矩阵上以提取独立信号； [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=D3VCCWPJ)“Kurtosis can be used as a measure of non-Gaussianity, and the extracted signal can be obtained by finding the unmixing vector which maximizes the kurtosis of the extracted signal” ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) 峭度可以用来评估信号的非高斯度，因此通过寻找让峭度最大的矩阵来提取混合的信号； 基于峭度的方法优点是容易计算，但是峭度对异常值十分敏感，因此该方法的鲁棒性不高 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=FN3JYD8U) ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) 峭度的计算公式 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=236&annotation=GB76BURN) ([Tharwat, 2021, p. 236](zotero://select/library/items/NAHTFCR3)) 对于白化后的信号，由于E(E^2)=1，因此峭度计算公式可以简化为上式 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=238&annotation=QFMY6Y4K)“Hence, different approximations have been introduced for calculating the negentropy” ([Tharwat, 2021, p. 238](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=BHHEDNWI)“independent components can be obtained by minimizing the mutual information between different components” ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 基于互信息度的方法通过最小化互信息度来获取独立成分 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=I27QXFBI) ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 两个变量之间的互信息度定义 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=89NU7C2H) ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 多个随机变量之间的互信息度 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=R25WHGKY) ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 从该公式可以看到，基于非高斯度的方法和基于互信息度的方法是相关的，两个仅相差符号和一个常数 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=B3QZKNV6)“non-Gaussianity measures enable the deflationary (one-by-one) estimation of the ICs which is not possible with mutual information or likelihood approaches.” ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 基于非高斯度的方法可以逐个提取独立分量，而这在基于互信息度和最大似然函数的方法中都是不可行的。 ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 非高斯性测量指标可以使得独立成分的逐个估计成为可能，这是互信息或似然方法所不能实现的。在ICA中，通常使用一种称为“迭代缩减”（deflationary）的方法，该方法一次估计一个独立成分。在每一次迭代中，使用非高斯性测量指标来选择最不高斯的信号作为下一个独立成分。这种方法通常比同时估计所有独立成分的方法更有效。 与互信息或似然方法相比，使用非高斯性测量指标具有更好的性质。互信息和似然方法通常需要一次性估计所有的独立成分，因此可能存在估计过程中的交叉干扰，导致估计结果不准确。而使用非高斯性测量指标则可以逐个估计独立成分，避免了这种交叉干扰，提高了估计的准确性。 常用的非高斯性测量指标包括峭度（kurtosis）、负熵（negentropy）等。峭度是描述分布偏斜程度的指标，对于高斯分布，峭度值为3，而对于非高斯分布，峭度值通常大于3。负熵是描述分布非高斯性的指标，它是信号熵与高斯分布熵之间的差值。在ICA中，通过最小化独立成分的峭度或最大化独立成分的负熵来选择最不高斯的信号作为下一个独立成分。 引用自chatgpt的回答 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=239&annotation=VEFH4X4L)“with the non-Gaussianity approach, Independent component analysis 23” ([Tharwat, 2021, p. 239](zotero://select/library/items/NAHTFCR3)) 同时，在基于非高斯度的方法中，各个独立分量被强制分解为不相关的，但是这个限制在基于互信息度的方法中并不是必须的 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=ZPC7ASLW)“all signals are enforced to be uncorrelated, while this constraint is not necessary using mutual information approach.” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=55CQGP3D)“Maximum likelihood (ML) estimation method is used for estimating parameters of statistical models given a set of observations” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=3NKP3UTC)“the likelihood and mutual information are approximately equal, and they differ only by a sign and an additive constant.” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) 基于最大似然估计的方法和基于互信息度的方法本质上是等价的，仅相差符号和一个常数 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=GN6HQK3Q)“maximum likelihood estimation will give wrong results if the information of ICs are not correct; but, with the non-Gaussianity approach, we need not for any prior information” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) 应该注意到，如果关于独立分量的先验知识是错误的，那么最大似然估计可能会给出错误的结果，但是基于非高斯度的方法并不需要任何先验知识。 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=DQEGJIVP)“Projection pursuit (PP) is a statistical technique for finding possible projections of multidimensional data” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) 投影追寻是一种用于寻找多维数据可能投影的统计学方法 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=AYTL8BTZ)“In the basic one-dimensional projection pursuit, the aim is to find the directions where the projections of the data onto these directions have distributions which are deviated from Gaussian distribution, and this exactly is the same goal of ICA [13]. Hence, ICA is considered as a variant of projection pursuit.” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) 在基本的一维投影追寻中，其目标是寻找一些列投影方向，在这些方向上，数据分布尽可能的偏离高斯分布，而这于ICA的目标一致，因此，ICA也会被认为是投影追寻的一个变种 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=240&annotation=DXKWW8D9)“In PP, one source signal is extracted from each projection, which is different than ICA algorithms that extract p signals simultaneously from n mixtures.” ([Tharwat, 2021, p. 240](zotero://select/library/items/NAHTFCR3)) 投影追寻中，独立分量是逐个被提取出来的。通过不断重复寻找使非高斯度最大的投影方向来逐个提取独立分量。这种方法被称为紧缩； [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=241&annotation=PDGRETG3)“In the projection pursuit algorithm, mixture signals are first whitened, and then the values of the first weight vector ðw1Þ are initialized randomly. The value of w1 is listed in Table 1. This weight vector is then normalized, and it will be used for extracting one source signal ðy1Þ. The kurtosis for the extracted signal is then calculated and the weight vector is updated to maximize the kurtosis iteratively.” ([Tharwat, 2021, p. 241](zotero://select/library/items/NAHTFCR3)) 在投影追寻算法中，混合信号首先被白化，之后我们随机初始化第一个权重向量，然后我们将该权重向量正则化，被提取一个源信号，之后我们计算提取出来的信号的峭度，被逐步更新以使峭度最大 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=241&annotation=DKJHA8XD)“FastICA algorithm extracts independent components by maximizing the non-Gaussianity by maximizing the negentropy for the extracted signals using a fixed-point iteration scheme” ([Tharwat, 2021, p. 241](zotero://select/library/items/NAHTFCR3)) FastICA算法通过一种固定点迭代的策略来最大化非高斯度从而提取独立分量 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=244&annotation=U95AFF5B)“FastICA has a cubic or at least quadratic convergence speed and hence it is much faster than Gradient-based algorithms that have linear convergence” ([Tharwat, 2021, p. 244](zotero://select/library/items/NAHTFCR3)) FastICA次或至少二次收敛速度,因此它比基于比梯度算法的线性收敛快得多 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=244&annotation=7RN663DZ)“FastICA has no learning rate or other adjustable parameters which makes it easy to use.” ([Tharwat, 2021, p. 244](zotero://select/library/items/NAHTFCR3)) FastICA算法没有学习率或其他任何需要调整的参数，因此很容易使用 [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=244&annotation=4L76QHDK)“FastICA can be used for extracting one IC, this is called one-unit, where FastICA finds the weight vector ðwÞ that extracts one independent component.” ([Tharwat, 2021, p. 244](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=244&annotation=S4VJ88FM)“Several units of FastICA can be used for extracting several independent components,” ([Tharwat, 2021, p. 244](zotero://select/library/items/NAHTFCR3)) [Go to annotation](zotero://open-pdf/library/items/APD2TRP8?page=244&annotation=FJ9PQEVA)“Deflation orthogonalization method is similar to the projection pursuit, where the independent components are estimated one by one” ([Tharwat, 2021, p. 244](zotero://select/library/items/NAHTFCR3))