In this paper a new framework, called Compressive Kernelized Reinforcement Learning (CKRL), for computing near-optimal policies in sequential decision making with uncertainty is proposed via incorporating the non-adaptive data-independent Random Projections and nonparametric Kernelized Least-squares Policy Iteration (KLSPI). Random Projections are a fast, non-adaptive dimensionality reduction framework in which high-dimensionality data is projected onto a random lower-dimension subspace via spherically random rotation and coordination sampling. KLSPI introduce kernel trick into the LSPI framework for Reinforcement Learning, often achieving faster convergence and providing automatic feature selection via various kernel sparsification approaches. In this approach, policies are computed in a low-dimensional subspace generated by projecting the high-dimensional features onto a set of random basis. We first show how Random Projections constitute an efficient sparsification technique and how our method often converges faster than regular LSPI, while at lower computational costs. Theoretical foundation underlying this approach is a fast approximation of Singular Value Decomposition (SVD). Finally, simulation results are exhibited on benchmark MDP domains, which confirm gains both in computation time and in performance in large feature spaces. A Random Projections is a simple but elegant technique that has both a strong theoretical foundation and a wide range of applications including signal processing, medical image reconstruction, machine learning and data mining. Its theoretical foundation rests on the Johnson–Lindenstrauss lemma [S in a high-dimensional feature space nR, if we construct an orthogonal projection of those sample points onto a random d-dimensional subspace, then if P > 1/2) up to a small distortion factor of 1 ± ɛ. Intuitively, this process can be thought of as applying a random rotation to a high-dimensional manifold and then reading off the first d coordinates. Compared with other linear dimension reduction methods, like Principal Component Analysis (PCA), Factor Analysis (FA), etc., Random Projections are data-independent, which significantly reduces the computational cost. In [This paper explores a technique called ss lemma : given a=Olog|S|ɛ, the procost. In  is analycost. In , Random L2 regularization is also intensively studied in these Gaussian process driving approaches and also in [Kernelized reinforcement learning, as it is named, aims at bring the benefits of non-parametric kernel approaches to reinforcement learning algorithms family. A kernelized least-squares policy iteration (KLSPI) is proposed in  by repla also in . In KLSP also in . KLSPI rIn this paper, a new framework, called Compressive Kernelized Reinforcement Learning (CKRL), for computing near-optimal policies in sequential decision making with uncertainty is proposed via incorporating the non-adaptive data-independent Random Projections and nonparametric Kernelized Least-squares Policy Iteration (KLSPI). One of the central ideas is that Random Projections are able to constitute an efficient sparsification technique and this brings about both faster convergence rate and lower computational costs than regular LSPI. Theoretical foundation underlying this approach is a fast approximation of Singular Value Decomposition (SVD). Experimental results also demonstrate that this approach enjoys the benefit of nonparametric approaches as well as alleviating the computation cost induced by non-adaptive random projections.Here is a brief roadmap to the rest of the paper. In Section 2, background knowledge of the three major perspectives comprising this paper is introduced on compressed sensing and random projections, kernel regression and sparsification and approximate Markov Decision Processes algorithms. In Section 3, unified framework and overview of state-of-art kernelized reinforcement learning algorithm is given, with extensive analysis on both kernel sparsification and error decomposition analysis. In Sections 4 and 5, the algorithms of Compressive Kernelized Reinforcement Learning algorithm are proposed with intensive theoretical analysis in Section 5. Finally, experimental results are conducted in the context of various experimental settings on different benchmark domains to validate the effectiveness of the proposed approach in Section 6.2.2.1.Let us first have a brief review of some important concepts and theorems that will be used in this paper.Lemma 1: (Restricted Isometry Property) A m × n compression matrix C ∈ Rm×n satisfies the Restricted Isometry Property (RIP), -RIP, if it acts as a near-isometry with distortion factor ε over all k-sparse vectors, that is, for any k-sparse vector x ∈ nR, the following near-isometry property holdsC whose columns have unit lengths. Each entry of C matrix ijc is draw from zero mean, unit variance distributions, which looks as if using random noise basis at the first glance, and exert a pairwise distance preserving projection that satisfies Johnson–Linderstrauss Lemma. Geometrically, RP is a simple geometric technique for reducing the dimensionality of a set of points in Euclidean space while approximately preserving pairwise distances with high probability P > 1/2 (w.h.p). If ɛ. From the sampling perspective, RP is performing coordinate sampling after a spherically random rotation to nR [d coordinates. RP is computationally efficient because of its data-independence nature, yet sufficiently accurate for dimensionality reduction of high-dimensional data sets.on to Rn . Intuiti2.2.K is a symmetric matrix with each entry ijK = k . If the kernel matrix K is not only symmetric but also positive semi-definite, Mercer’s theorem states that the kernel function can be interpreted as an inner product of nonlinear function ϕ  between every pair of instances without explicitly knowing the nonlinear function’s form. Kernel trick is application of Mercer’s theorem to enrich the expressiveness of the feature space without manually adding more features. Geometrically, kernelization is a way of obtaining linearity by practising a high-dimensional embedding through a nonlinear mapping: First, an inner product is introduced by defining the “nonlinear” kernel, then the whole space is embedded into the high-dimensional kernel space with inner-product preserving, and finally linearity is obtained in this high-dimensional kernel space.Now we give a brief introduction of kernel, kernel matrix, kernel trick and kernel regression. A kernel is a symmetric function representing the similarity between two samples,t = Φw + ε, the sum-of-squares error function without L2 regularization term is given by w = ΦTa, and Gram matrix K = ΦΦT, we have t represents the target values of the sample points, and k(x) is a column vector where ik (x) = k . The problem with this method is that there is no guarantee that the kernel matrix is invertible. The most common remedy for this problem is to introduce ridge regression/L2 regularization into this kernelized version, adding regularization term λI to the diagonal so that K will be invertible. Thus (3) would becomeK while preserve its rank so that it is more likely to be nonsingular as its dimension decreases. ALD method can be considered as a tentative approach in this category. Another approach is to try to practise the pairwise-distance preserving compression, e.g., using Random Projections for matrix compression. We hereby give a brief review of Compressed Linear Regression by [C = m,nC. Firstly we introduce compression matrix d,nC to linear model, so there isL2 regularization term is given byKernel regression , also cassion by . SupposeTCKC is nonsingular whenever rank(K) ≥ d.We now give proof that the compressed kernel matrix Theorem 1: If the kernel matrix K satisfies rank(K) ≥ d, the compressed kernel TCKC will be nonsingular, i.e., rank(TCKC) = d, where C is the randomly generated compression matrix.Proof:rank(K) ≥ d, there exists a sequence {ir1, ir2, ⋯, rdi} and {ic1, ic2, ⋯, cdi} such that the sub-matrix subK which are drawn from K with rows of index {ir1, ir2, ⋯, rdi}, and columns of index {ic1, ic2, ⋯, cdi} such that rank (subK) = d Next draw arbitrary d rows and columns from C, which form square matrix subC such thatC is approximately orthogonal, so is subC. Also since the rank of a matrix is invariant by left-multiplying a full column-rank matrix or right-multiplying a full row-rank matrix, we have rank (TCKC) = d. This explains why the compressed kernel matrix TCKC is more prone to be nonsingular compared with the original kernel matrix K, which provides an alternative way besides ridge regression to handle K if it is ill-conditioned.If 2.3.Markov Decision Process (MDP) [S, A, R, γ), comprised of a set of states S, a set of (possibly state-dependent) actions A (sA), a dynamical system model comprised of the transition probabilities s′ from state s under action a, and a reward model R. A policy π: S → A is a deterministic mapping from states to actions. Associated with each policy π is a value function πυ, which is a fixed point of the Bellman equation:γ < 1 is a discount factor. In what follows, we often drop the dependence of πυ on π, for notational simplicity. When the set of states S is large, it is often necessary to approximate the value function v using a set of basis functions . In linear value function approximation, a value function is assumed to lie in the linear span of a basis function matrix Φ of dimension S × k, where we assume that k ≪ |S|. Hence, υ ≈ υ̂ = Φw. The vector space of all value functions is a normed inner product space, where the “length” of any value function f can be measured with respect to a weighted norm ξ asξ. It is often the case that ξ is selected to be the invariant distribution of the Markov chain induced by a policy π, where the distribution is assumed to be ergodic. For example, one approach to approximately solving the Bellman equation is the fixed point (FP) solution, often referred to as the TD approach, which is defined as finding a set of weights FPw such thatΦ is the orthogonal projection onto the column space of Φ, given bycombined projected back-up operator ΠΦT. Bellman residual methods, in contrast, find a set of weights BRw under which the difference between the approximated value function and the back-up approximated value function T(Φw) is minimized. That is,L1-based feature selection method, i.e., LARS, LASSO, etc. Compressed Sensing and Random Projections provide another path to tackle this problem as an alternative to L1-regularization, e.g., the work presented in [L1-regularization methods.A ss (MDP)  is definented in , which pi.e., samples are not used only once (as in Q-learning), but for multiple times instead. Since samples would be scarce in the large feature space, data reusage is of critical importance.In Reinforcement Learning, it is indeed a huge computational challenge for LSPI to work with large amount of features. Besides heavy computation costs, another concern is learning performance, since a lot of training data is needed for large feature spaces. The third concern is data efficiency. Often the key to data-efficiency is sample reusage, L1 based method into LSPI, which is called LARS-TD [L1 regularization and feature selection. Another way is to do feature compression with Random Projections as an alternative of feature selection, as proposed in [Corresponding to the methods mentioned above, introducing  LARS-TD , provideposed in , which iposed in , a kerneposed in  on kerne3.3.1.Nonparametric approximators have been combined with a number of algorithms for approximate policy evaluation. For instance, kernel-based approximators are combined with LSTD and LSTD-Q by ,11, and i.e., the number of samples that contribute to the solution ..i.e., thKernel-based LSPI introduces the kernel trick into least-square temporal difference learning to derive a kernelized version of LSTD. In KLSPI, kernels are used as basis in LSPI framework for efficient policy evaluation. A generalized kernelized model-based LSTD framework is presented in , where kK′ = PK, whereK, K′ is not symmetric. K′ can be thought of kernelized model of transition matrix P. The kernel regression of the reward model isis, ia, ir, s′i, π (s′i)) are drawn from an identical trajectory, we haveG is a nil-potent upper shift matrixLet us define 3.2.L2-regularized kernelized LSTD, which can be roughly divided into three categories to the best of our knowledge. The first is adding regularization term to the kernel regression model of (12) and (13), respectively, i.e.,R and ΣP are L2-based regularization terms of reward model and transition model, respectively. Further discussions regarding the corresponding value function formulation and relations between the two regularization term ΣR, ΣP is in [How to successfully implement kernelized reinforcement learning algorithm involves various sparsification and regularization method. There are two main questions concerning the regularization. The first question is “How to regularize?”. To answer this question, there are several frameworks of ΣP is in . Gaussian is constructed, where n is the number of samples, which uses kernel values between pairs of next state. The authors have shown that the algorithm can be used efficiently when the value function approximation lie in a reproducing kernel Hilbert space (RKHS). They also developed finite sample error bound for the regularized algorithm.Another category of regularized kernelized LSTD is in , in whicLemma 2 [Lemma 2 : The KLSProof: We give a sketch proof here which extends mainly three steps. In a same trajectory, s′i = is+1, so we have K′ = GK. Secondly using K′ = GK we can haveThe third sparsification technique is aiming at reducing sample complexity in streaming data, which renders it applicable in real applications. A sparsification procedure based on approximate linear dependency (ALD) can be performed, which is an online, fast approximate version of PCA . ALD reapreprocessing and postprocessing. Preprocessing is to directly compress the feature space by feature selection. It then learns a basis of the sample  from the compressed feature space, and then use it in LSPI. In this case, it is equivalent to compressing the feature set in advance, and the same feature vector is used at every iteration for a new sample. The ALD sparsification is a typical method of this approach. In this method, each sample in the compressed dictionary spans a feature, and ALD is aiming at compressing this dictionary so that the number of elements in the dictionary is much smaller compared with the number of the whole sample set. In the ALD approach, a subset of samples x̃1, ..., dx̃ is constructed in order to avoid redundant information. The dictionary is used such that ϕ(x̃1), ..., ϕ(dx̃) spans approximately ϕ(x1), ..., ϕ(nx), while being of minimal size. So the sparsification is actually in the feature space and can be counted as doing feature selection via abandoning redundant samples in forming the dictionary. KLSPI is an adaptation of this idea into LSTD framework. In [Except the first question on how to implement to regularization, another question, both palpable and profound, is when to implement regularization. Generally there are two possible sparsification schemes which differ in when to practise sparsification: work. In , ALD is indirectly adapt sparsification to LSTD, which is to learn a high-dimensional basis k̃ (is) of the sample ) first, then use some technique for sparsification, and use the sparsified basis k (is) in policy iteration to generate the new policy. In this case, one needs to recompute a basis at each iteration of LSPI since different feature vectors are used at each iteration, as π (s′) depends on the current policy. The A, b matrix is computed as follows:The postprocessing scheme is to 3.3.R and the transition matrix P, respectively. The geometric illustration can be seen in Let us move to introduce the Bellman error of the kernelized value function, which is the one-step temporal difference error,rding to , the Bel4.The General Framework of KLSPI with sparsification is described in 5.C to stand for compressed version, K for kernelized version, CK for kernelized LSTD-RP, and KC for compressed KLSTD. In the following, n stands for the number of samples, D stands for the dimension of full dimensional space, and d stands for the dimension of compressed dimensional space. In kernel regression, we have D = n.We use subtitle Here is using ALD as an optional pre-processing sparsification step and LSTD-RP as a post-processing sparsification step.5.1.C is generally not orthogonal . A linear mapping can cause significant distortions in the data set if C is not orthogonal. Orthogonalizing C is, however, usually very computationally expensive and thus does not justify its cost. Instead, we can rely on a result in [almost orthogonal than orthogonal directions. Thus vectors having random directions might be sufficiently close to orthogonal, and equivalently T CC ≈ I, where I is the identity matrix, and according to experimental experience [T CC and an identity matrix was about We have to be careful to extend any conclusion of linear projection to Random Projections since there is no guarantee that the Random Projections matrix is a projection operator, because esult in , that isperience , the mealemma in  can be uTheorem 2 Suppose that A is a real m × n matrix. Select a target rank k ≥ 2 and an oversampling parameter p ≥ 2, where k + p ≤ min {m, n}. Execute the proto-algorithm with a standard Gaussian test matrix to obtain an m × (k + p) matrix Q with orthonormal columns. Then the expectation of approximation error is boundedProposition 1: For compression matrix C, the following holds:C is approximately orthogonal.Each column of TCC formulates an approximately identity matrix, i.e., T CC ≈ I.etc. Principal Component Analysis, as a unsupervised dimensionality reduction method, is the optimal solution when the target cost function is the sum of the mean square reconstruction error. The general idea of PCA, kernel PCA and ALD is to project the entire ambient feature space onto a lower dimensional manifold spanned by the topmost eigenvectors of the sample covariance matrix in feature space corresponding to the leading eigenvalues.Low rank matrix approximation is important in a wide variety of scientific applications including statistics, signal processing, machine learning, X to minimize the approximation error of ||A − X||. If we limit the minimizer to the form of X = T ACC, the problem formulation of this family would be divided into fixed-precision approximation problem and fixed-rank approximation problem [The problem formulation is trying to find a low rank matrix  problem .fixed-precision approximation problem, suppose we are given a m × n matrix A and a positive error tolerance ε. The task is to find a compression d × m matrix C with minimal d = d (ε) orthonormal column such thatL2 norm. The range of C is a d–dimensional subspace that captures most of the actions of A, and d is desired to be as small as possible.For the fixed-rank approximation problem, given a matrix A, a predefined compression rank d, the problem is to find matrix C with orthonormal columns such that the approximation error ε = ε (d) is minimized.fixed-precision problem, i.e., given a matrix A, a target compression rank d, the problem is to findC and the topmost d dominant left singular vectors of A, specifically, X = TUSV, where dU is of size n × d and contains these d singular vectors, namely,A in the compressed d-dimensional subspace.For 5.2.A, b is defined in [CV̂ (s) is represented asFirst let us have a brief review of LSTD-RP. In LSPI-RP, there isfined in ,(25)A=Φ5.3.i.e., if we perform Compressed Linear Regression to the sample set, and practise Kernel-based LSTD based on the compressed kernel matrix, this would be identical to the performance of the kernelized version of LSTD-RP for feature compression.Next we will prove the equivalence of the compressed Kernel-based LSTD with kernelized LSTD-RP, Theorem 3: The solution of the kernelized LSTD-RP is the approximate solution of the compressed kernel-based least-square temporal difference learning algorithm, namely,Proof:(1): For kernelized LSTD-RP, we just replace the basis set Φn,D with n,nK in n = D. Thus we getCKV̂ is,(2):Next we develop the compressed version of kernelized value function approximation. As in [KV̂ can be represented asn. As in , the ker(3): Based on the above analysis, we haveKCV̂ isNote that if we insert 6.6.1.C; it is thus reasonable that the variance will be higher than in the ALD-based methods. Also we restrict the problems with finite sample set and large amount of features.We now present several comparison studies to show the effectiveness of our method. It is noteworthy to mention that the benefit of introducing Random Projections for feature compression lies mainly in reduction of computational cost. The overall performance, however, highly depends on the quality of the randomly generated compression matrix L1 regularization, LSPI-RP does not assume that the representation w.r.t the original basis is sparse. To implement L1-regularized method, we need the target function to be sparse in the selected representation. Mathematically, the sparsity appears in the bound instead of the dimension. So, if the signal is not sparse, its sparsity will be equal to the dimension, and thus, the large dimension appears in the bound. In LSPI-RP, however, instead of the requirement on sparsity of the original basis, a requirement is imposed on the features such that they are supposed to be of specific form in order to perform better than LSPI in high-dimensional space. So in LSPI-RP it would be of critical importance of the basis [F, which would often lead to overfitting for regular LSPI. The setting is preferable for LSPI-RP where the number of samples is smaller than or at the same order of the number of features.Unlike he basis . The sec6.2.d: the compression dimensionn: the number of samples, which is also the full dimension in kernelized LSPIθ: the compression ratio of d/D.ɛ: the threshold parameter for ALD dictionary.First we define parameters used in the experiment:The pendulum domain is a typical under-actuated system which is highly unstable. Here we give some illustrative example of compressed kernelized LSPI in pendulum domain. Here we collect 2, 000 samples, and the compression ratio is 0.2. Here we take 20 runs on average, and the average runs to converge of compressed KLSPI, ALD-based KLSPI and regular LSPI are 7.4, 7.8, 8.6 runs. From here we can see that compressed KLSPI has the advantage of faster convergence. This merit will be more explicit as we will show in θ1 and θ2, and the angular velocities, θ̇1 and θ̇2. There are three actions corresponding to positive (a = 1), negative (a = −1), and zero (a = 0) torque. The time step was set to 0.05 and actions were selected after every fourth update to the state variables according to the equations of motion. The goal for this domain is to raise the tip of the second link above a certain height in minimum time . The reward function is therefore −1 for each time step until the goal is achieved and the discount factor is γ = 0.99. Episodes begin with the all state variables at value 0 which corresponds to the two links hanging straight down and motionless. The Acrobot is an under-actuated double pendulum which is a typical working benchmark for its nonlinear dynamics. It consists of two arms where torque can only be applied at the second joint. The system is described by four continuous variables: the two joint angles, d with different number of samples. Up to our best knowledge, there is no tighter bound on how to choose d in compressed LSPI yet. In [d is given with an empirical extension to a rule of thumb that In this experiment, we shall seek the optimal compressing dimension  yet. In , a coverε. Likewise, the compressed sensing technique also depends on one parameter: the compressed ratio θ = d/D. To be fair, we will give identical compression dimension d, which means that the compressed dimension is equal to the size of the ALD dictionary. The problem setting in Experiment 2 is like this: given large amounts of data, the ALD dictionary is still considerably large. How to compress this dictionary further is an interesting topic worth attention. One heuristic idea, of course, is to make the accuracy threshold of ALD dictionary υ larger so that the accuracy tolerance becomes larger, and correspondingly the dictionary size becomes smaller. However, based on the compressive reinforcement learning framework, here is another attractive method, first we build the ALD dictionary with a very strict accuracy threshold υ and build an ALD dictionary with large size. Then we use the Random Projections to compress the basis generated by this “large” dictionary. Comparison studies are carried out and experimental results in In this experiment, several comparison studies are carried out to compare the two kernel sparsification technique: ALD and Compressed Sensing. Briefly, both two methods are highly adaptable and only depends on one parameter. The ALD method depends mainly on one parameter: the dictionary threshold 6.3.In this experiment, we will show the approximated value function of Compressed Kernelized LSPI on pendulum domain. From 6.4.In here we give a new kernel function, namely, Geodesic Gaussian Kernel (GGK). Compared with RBF kernel, GGK can capture the topology on the manifold where the samples lies on instead of the Euclidean distance used in RBF kernel. Moreover, since GGKs are local by nature, the ill effects of local noise are constrained locally.Geodesic Gaussian Kernels on Graphs:SP  denotes the shortest path from state s to state s′. Since the shortest path on the graph can be interpreted as a discrete approximation to the geodesic distance on a nonlinear manifold, geodesic Gaussian kernel (GGK) includes rich information of the manifold formed by the data. Shortest paths on graphs can be efficiently computed using the Dijkstra algorithm. The result for two-room domain using GGK is show in A natural definition of the distance would be the shortest path. So we define Gaussian kernels on graphs based on the shortest path:7.L1 regularization into this framework is a promising topic to explore. Another promising future direction is to introduce this compressive reinforcement learning framework to policy gradient approaches with regular MDP [In this paper a new nonparametric reinforcement learning framework called Compressive Kernelized Reinforcement Learning (CKRL) is proposed based on Gaussian process, compressed Sensing and random projections. We compare compressed kernelized LSPI, kernelized LSPI, and regular LSPI along with different kernels based on Euclidean distance and Graph-based Geodesic distance, respectively. Preliminary theoretical proof and experimental results are also given. There are various interesting future directions along this research venue. For instance, how to integrate ular MDP  and POMDular MDP .


In this paper, the potential sensitivity in Kelvin probe force microscopy (KPFM) was investigated in frequency modulation (FM) and heterodyne amplitude modulation (AM) modes. We showed theoretically that the minimum detectable contact potential difference (CPD) in FM-KPFM is higher than in heterodyne AM-KPFM. We experimentally confirmed that the signal-to-noise ratio in FM-KPFM is lower than that in heterodyne AM-KPFM, which is due to the higher minimum detectable CPD dependence in FM-KPFM. We also compared the corrugations in the local contact potential difference on the surface of Ge (001), which shows atomic resolution in heterodyne AM-KPFM. In contrast, atomic resolution cannot be obtained in FM-KPFM under the same experimental conditions. The higher potential resolution in heterodyne AM-KPFM was attributed to the lower crosstalk and higher potential sensitivity between topographic and potential measurements. Kelvin probe force microscopy (KPFM)  combinedz2. AM-KPFM, proposed by Kikukawa et al. [z, and a serious stray capacitance effect [On the other hand, there are mainly three kinds of KPFM modes: frequency modulation (FM), amplitude modulation (AM) , and heta et al. , has dema et al. ; howevere effect ,15. As ae effect  and Ma ee effect , has beeConsequently, to elucidate the origin of atomic resolutions of potential measurements in FM, AM, and HAM-KPFMs, it is necessary to clarify the performance of topographic and potential measurements using the three modes. Here, since the serious stray capacitance effect on LCPD images in AM-KPFM has been illustrated in the past , we simpIn this study, for the first time, we investigated HAM-KPFM as a method of enabling quantitative surface potential measurements with high sensitivity by showing the contrast between FM- and HAM-KPFMs. The principle and experimental setup of FM- and HAM-KPFMs are presented. The high sensitivity of HAM-KPFM compared to FM-KPFM is experimentally demonstrated. Finally, we gave atomic resolution images of surface potential measurements on a Ge (001) surface using a W-coated cantilever in HAM-KPFM.f) in cantilever vibration and the energy dissipation results in an amplitude variation (∆A) of the cantilever's oscillation; these parameters are given by △f = - f0Fc/(2kA), △A = QFd/k[f0, k, Q, and A are the resonance frequency, the spring constant, the quality factor, and the amplitude of the cantilever, respectively. Fc and Fd are the tip-sample conservative and dissipative interactions, respectively.Firstly, we theoretically compared the performance of potential sensitivities in FM- and HAM-KPFMs. In NC-AFM, the frequency shift  between the tip and the sample, ε0 and R are the dielectric constant in vacuum and the tip radius, respectively. zt0 and A are the average tip position and the oscillation amplitude of the cantilever, respectively.here, Direct current (DC) component of the frequency shift induced by alternating current (AC) bias voltage is given by:From the equation ribed by δVCPD-FM is approximately 15.11 mV with a VAC of 1 V. That means that if we want to obtain a potential resolution higher than 15 mV, VAC has to be higher than 1 V.Note that the minimum detectable CPD in FM-KPFM is independent of the quality factor of the cantilever. Under the typical conditions in Table VBias = VDC + VACcos (ω2 - ω1) t is applied; the electrostatic force at frequency ω2 is given byIn HAM-KPFM, a bias voltage ω2 are described byamplitude variation induced by the electrostatic forces at frequency A = QFd/k, the minimum detectable CPD is given byfrom the equation △VCPD-HAM is approximately 5.52 mV with a VAC of 1 V. This value is around three times smaller than that of δVCPD-FM. In other words, to achieve an equivalent potential resolution, the VAC in HAM-KPFM is smaller than that in FM-KPFM.Note that minimum detectable CPD in HAM-KPFM is inversely proportional to the quality factor of the cantilever. Increasing the quality factor of the cantilever decreases the minimum detectable CPD, which means that the potential sensitivity in HAM-KPFM is enhanced. Under the typical conditions in Table These results show that the potential and force sensitivity detected by HAM-KPFM is higher than in FM-KPFM especially with the higher quality factor of the cantilever in vacuum condition.Next, we experimentally confirmed that the potential sensitivity of HAM-KPFM is higher than that of FM-KPFM. All experiments were performed with homemade optical interference detection UHV-AFM equipment operating at room temperature. FM-AFM was performed to provide topographic and dissipation information. The frequency shift was fed into the SPM controller  as feedback to keep it constant; data acquisition and distance spectroscopy were performed by the Nanonis system.ωm component of the frequency shift for FM, and ω2 component of the cantilever deflection for HAM-KPFM, respectively, which was generated by the lock-in amplifier into the SPM controller. The FM- and HAM-KPFM setup diagrams are shown in Figure ACcos (ωmt) which was generated by the commercial phase-locked-loop detector was applied between the tip and the sample, the ωm component of the frequency shift Δfm is measured with the PLL circuit and the lock-in amplifier. In HAM-KPFM, an AC bias voltage of VACcos (ω2 - ω1) t was applied between the tip and the sample, the ω2 component of the cantilever deflection is measured with a lock-in amplifier . The details of the experimental setup have been given in references [Simultaneous measurements of the potential information (LCPD) were measured by FM- and HAM-KPFM, respectively. The DC bias voltage was tuned to minimize the electrostatic interaction with the bias feedback by feeding the ferences ,12.k, resonant frequency f1st, and quality factor Q of about 40 N/m, 165 kHz, and 20,000, respectively, and where f2nd is approximately 6.3 times higher than f1st (f2nd ≈ 1.05 MHz). The modulation frequencies in FM- and HAM-KPFM were fmod-FM = 500 Hz, fmod-HAM = f2nd = 1.05 MHz. The cantilever was initially treated with an Ar+ ion bombardment  to remove the native oxidized layer and maintain tip sharpness. The tip was then coated by a tungsten layer with a thickness of several nanometers by sputtering the tungsten mask plate for 10 h  to ensure sufficient tip conductivity [+ ion sputtering at 1 keV followed by annealing to 973 to 1,073 K.A commercial silicon cantilever (Nanosensors: NCLR-W) which was detected by an optical fiber interferometer was used as the force sensor, with a spring constant uctivity . A Ge  and (8), one can find that the minimum detectable CPD in HAM-KPFM is 1/3 that obtained in FM-KPFM in theory, in contrast, the SNR in HAM-KPFM is just 1.5 times higher than that in FM-KPFM. A possible explanation for this difference comes from the fact that quality factor of the cantilever we used was less than the simulation one. The SNR of FM-KPFM results at VAC = 500 mV is consistent with the measurement result in literature [We compared the signal-to-noise ratios (SNRs) of detected signals at different bias modulation amplitudes to investigate their sensitivities to short-range electrostatic force in FM- and HAM-KPFMs. Figure terature .VAC = 150 mV, ∆f = -68.5Hz, and modulation frequencies in FM- and HAM-KPFMs are identical to the previous SNR measurements, respectively. The scanning area was 4 nm × 4 nm.We have taken potential distribution measurements on a Ge (001) surface by FM- and HAM-KPFMs using the system mentioned above. The cleaned Ge (001) surface showed a buckled dimer structure with a low, missing-dimer defect distribution. There are two main buckled dimer structures: the symmetric dimer phase p (2 × 1) configuration and the c (4 × 2) configuration ,19. ThisVDC approximately at 1 V) and this voltage easily makes the dimer atoms on the surface adsorbing to the tip apex to form double covalent bonding with the surface atoms. Besides, the influence of the topographic measurement seriously affects the potential images with high AC bias voltage. In contrast, for HAM-KPFM, this phenomenon can be ignored (the results are not shown here).These results demonstrated that the HAM-KPFM has a higher potential resolution and lower crosstalk than FM-KPFM.Figure VAC than that of FM-KPFM. The higher potential sensitivity of HAM-KPFM was explained as follows: the oscillation of the frequency shift at ω1 in FM-KPFM and the oscillation of the amplitude at ω2 in HAM-KPFM are both proportional to the gradient of the electrostatic force, whereas the quality factor in UHV for the AFM system is approximately several tens of thousands greater, and finally, that the minimum detectable electrostatic force in HAM-KPFM is smaller than in FM-KPFM according to Equations (1) and (2). Hence, the potential sensitivity in HAM-KPFM is higher than that in FM-KPFM. Further, lower crosstalk between topography and potential images in HAM-KPFM compared to that in FM-KPFM is due to the first and second resonance signals being separated from each other using low- and high-pass filters in HAM-KPFM; on the other hand, the potential and topography signals are difficult to separate because the first resonance of the cantilever was oscillated in both measurements.Quantitatively, the potential line profile contrast is shown in Figure VAC effect was apparently removed because small AC bias voltages were applied and the VCPD which compensated the CPD between tip and sample is 20 to 100 mV [VAC influence to the potential measurement [In HAM-KPFM measurements, the high o 100 mV ,12, and o 100 mV . In somesurement .In summary, the potential sensitivity and crosstalk were compared in FM- and HAM-KPFM experimentally and theoretically. We demonstrated that the potential sensitivity in HAM-KPFM is higher than that in FM-KPFM theoretically. Then, we experimentally confirmed that SNRs of electrostatic force measurements, which determined the potential sensitivity in HAM-KPFM, are higher than that of FM-KPFM. Further, we applied the FM- and HAM-KPFM measurements to a Ge (001) surface under the same conditions, and atomic resolution in potential and topography images were obtained in HAM-KPFM, whereas the atomic resolution was not visible in FM-KPFM. We attribute this to the higher sensitivity and lower crosstalk in HAM-KPFM compared to the FM-KPFM. Consequently, the HAM method proposed here is a useful tool for detecting the actual potential distribution on the surface.The authors declare that they have no competing interests.ZM, JM, JT, HX, and HZ carried out the calculations, performed the experiments, and drafted the manuscript with the help of CX and JL. YL participated in the design of the study and helped to draft the manuscript. All authors read and approved the final manuscript.


According to ideomotor theories, intended effects caused by a certain action are anticipated before action execution. In the present study, we examined the question of whether action effects play a role in cued task-switching. In our study, the participants practiced task-response-effect mappings in an acquisition phase, in which action effects occur after a response in a certain task context. In the ensuing transfer phase, the previously practiced mappings were changed in a random, unpredictable task-response-effect mapping. When changed into unpredictable action-effects, RT as well as switch-costs increased, but this occurred mainly in trials with short preparation time and not with long preparation time. Moreover, switch costs were generally smaller with predictable action-effects than with unpredictable action-effects. This suggests that anticipated task-specific action effects help to activate the relevant task-set before task execution when the task is not yet already prepared based on the cue. The question of how human actions are mentally controlled is one that has been investigated since the early beginnings of cognitive psychological research in the nineteenth century , to shift attention to the new, currently relevant task-set, and to activate the new task-set  and error rate] is worse after a task switch than after a task repetition. These Action effects, that is, effects that happen as a consequence of the specific task execution, might help to reduce the conflict of ambiguous task-sets. When referring to action effects, we are referring to effects or events that take place after the response, as explicit consequence of the response. Examples for action effects are the tone that is heard after pressing a piano’s keyboard or the light that is turned on after pressing the light switch.anticipation of the effect, which happens temporally before response execution, as assumed in the ideomotor principle. For example, when playing a piano, the pianist anticipates already the tone he/she wants to produce before pressing the key  effect anticipation.Kiesel and Hoffmann  reportedand vertically aligned. A cue indicated if the position of the horizontal or the vertical target should be determined. The task-related action effect was a red light-up of the target in the position which indicated the correct response. This means, if the correct response was “right,” the target on the right side appeared red as consequence of the response, whereas the vertical target stayed colorless; and if the correct response was “up,” the target turned red on the up-position, whereas the horizontal target stayed colorless. The action effects were thus semantically associated with the correct response. The task-unspecific effect was just a feedback with the information whether the task was executed correctly or not. The authors found a significant two-way interaction of task transition and type of effect for trials with a long cue-target interval . In the task-specific effect condition, residual switch costs were reduced compared to the unspecific effect condition. That is, anticipating task-specific action effects might help to discriminate the task sets of the upcoming trial and select the appropriate one. However, in this study, only trials with a long CTI were analyzed so that no statement can be made about how action effects influence task performance in trials with a short CTI.In another study, Ruge et al.  providedbefore response execution , we assume that they play a stronger role primarily when the task is not yet fully prepared .If the influence of anticipated action-effects needs time to build up, we would assume an influence of task preparation time. To examine the influence of preparation time on the impact of action effects in task-switching, we manipulated the cue-target interval. If the task-response-effect association becomes stronger with more time to activate the task set, then we would expect a more pronounced increase in RT and switch costs for long preparation intervals compared to short preparation intervals. However, with the presentation of the cue, the cue-task association is as well retrieved, entailing a sustained bias to the relevant task components Meiran, , so thatTwenty-four students of the RWTH Aachen university  took part in the experiment. They received partial course credit or 8€. The participants were equally and randomly assigned to the experimental vs. the control group. Informed consent was obtained from all subjects, and the experiment was performed according to the ethical standards of the Declaration of Helsinki.The experiment was programmed with the experimental runtime system ERTS . Participants sat in front of a screen with a viewing distance of approximately 60 cm. The stimuli consisted of digits ranging from one till nine, without the five. They appeared in white on a black background at the center of the screen with a height of 1.4 cm .The two tasks were two numerical judgment tasks. In one task, the participants had to decide if a presented number was greater or less than five . In the other task, the participants had to decide if the presented number was odd or even . The tasks switched randomly and were indicated by a cue. Each number was framed by either a diamond or a rectangle. If the number was framed by a diamond, the magnitude task was required; if the number was framed by a rectangle, the parity task was required. The diamond was 3 cm high and 3 cm wide . The rectangle was 3.6 cm high and 3.5 cm wide . Responses were to be made by manually pressing one of two response keys  with the left or right index finger. Participants were instructed to press left for a smaller or an odd number and to press right for a greater or an even number. If the response given by the participants was correct, an action-effect occurred. The effects were assigned to the task and the response in the experimental group in the first eight “acquisition blocks.” For example, for the magnitude task, visual action-effects and for the parity task, auditory action-effects occurred. If the response was “less” (left key-press), the background of the screen turned yellow and if the response was “greater” (right key-press), the background of the screen turned green. For the parity task, a honking tone was presented after the correct response for “odd” (left key-press) and a ringing tone after the correct response for “even” was presented (right key-press). The mapping between task, response, and action-effect was counterbalanced across participants in the experimental group. For the control group, the action effects were completely random, that is, participants could not establish an association between task, response, and action-effect and thus could not anticipate the action effect. If the response was wrong, nothing happened, the background just turned black in both groups. Participants were instructed to react as fast and as correct as possible. They were informed in both groups that the effects happening after the key-press should be used as feedback whether the task was performed correctly.Each trial started with a cue, which appeared in half of the trials 100 ms prior to the target stimulus (CTI of 100 ms) and in the other half of the trials 900 ms prior to the target stimulus. The CTI duration varied randomly. Cues and stimuli stayed on the screen until a response was given. For both CTI levels, an action effect was presented for 200 ms immediately after a correct response. That is, the response-cue interval (RCI) was held constant at 400 ms and the response-stimulus interval (RSI) was 500 ms for trials with a short CTI and 1300 ms for trials with a long CTI.Before the experiment started, the participants performed a practice block with 16 trials. The experiment consisted of eight acquisition blocks and one transfer block. The acquisition blocks as well as the transfer block comprised 96 trials each plus four warm-up trials that were not recorded. Altogether, one session lasted about 45 min.In the experimental group, the task-response-effect associations were predictable so that a mental task-response-effect association could be established by the participants. In the transfer block, the learned associations were changed into random action-effects as in the control group. There was no difference between the acquisition blocks and the transfer block in the control group. To analyze the influence of the change in the action-effect assignment, performance in the last acquisition block (Block 8) was compared with that in the transfer block (Block 9).Task transition (switch vs. repetition), block (Block 8 vs. Block 9), and CTI (short vs. long) were used as independent within-participants variables; group  was used as independent between-participants variable. RT and error rate were measured as dependent variables. Significance was tested at α = 0.05.In addition to the four non-registered warm-up trials, the first recorded trial of each block was excluded from analysis because it could not be categorized as a switch or repetition trial. All trials with RT less than 200 ms or exceeding 3 SD of each participant’s mean were discarded as RT outliers . Furthermore, all incorrect trials and that following an incorrect trial were not included in the analysis. Errors occurred in 4.8% of the trials.F = 5.49, p = 0.029, η2 = 0.2  = 4.98, p = 0.036, η2 = 0.185. Only in the experimental group, RT increased in Block 9 (from 817 to 882 ms), but not in the control group (776 vs. 775 ms). This interaction is also reflected in the split ANOVA, as described below.The data were submitted to a four-way mixed analysis of variance (ANOVA) with the independent variables task transition, block, CTI, and group. To forestall the most important result: the four-way interaction of task transition, block, CTI, and group was significant, F, 22 = 5.experimental group, the three-way interaction of task transition, block, and CTI was significant, F = 12.34, p = 0.005, η2 = 0.529. Switch-costs increased significantly from Block 8 (75 ms) to Block 9 (226 ms) in short CTI trials, whereas they even somewhat decreased numerically from Block 8 (61 ms) to Block 9 (49 ms) in long CTI trials. Further, the two-way interaction of task transition and CTI was significant, F = 6.7, p = 0.025, η2 = 0.378, showing that switch-costs decreased from 150 ms for short CTIs to 56 ms for long CTIs. The main effects of task transition, F = 30.03, p < 0.001, η2 = 0.732, block, F = 10.3, p = 0.008, η2 = 0.484, and CTI, F = 11.05, p = 0.007, η2 = 0.501 were significant, too. RT was shorter for repetition trials (798 ms) than for switch trials (901 ms), for Block 8 (817 ms) than for Block 9 (882 ms) and for long CTIs (807 ms) than for short CTIs (893 ms). To go more into detail, and understand better the three-way interaction, we further split the analysis in two two-way ANOVAs for short and long CTI trials.For the F = 6.75, p = 0.025, η2 = 0.380, indicating that switch-costs increased in the transfer block. Further, the main effect of task transition was significant, F = 35.38, p < 0.001, η2 = 0.763 whereas the main effect of block was just not significant, F = 4.13, p = 0.067, η2 = 0.273. For the trials with a long CTI, the two-way interaction of task transition and block was not significant, F < 1. The main effect of task transition failed to reach significance, F = 4.07, p = 0.069, η2 = 0.270, but the main effect of block was significant, F = 11.26, p = 0.006, η2 = 0.506. RT was increased from Block 8 (770 ms) to Block 9 (843 ms).The trials with a short CTI showed a significant two-way interaction of task transition and block, control group, the three-way interaction of task transition, block, and CTI was not significant, F < 1. The well-known two-way interaction of task transition and CTI was significant, F = 11.03, p = 0.007, η2 = 0.501. It reflects a reduction of switch costs with a long preparation interval . The main effects of task transition, F = 20.74, p = 0.001, η2 = 0.653, and CTI, F  = 47.69, p < 0.001, η2 = 0.813 were significant. RTs were faster in repetition trials (707 ms) than in switch trials (844 ms) and faster with a long preparation interval (709 ms) than with a short preparation interval (841 ms). There was neither a main effect of block, nor an interaction including this variable (Fs < 1).In contrast to the experimental group, in the F = 20.02, p < 0.001, η2 = 0.476. Switch costs were higher in the control group (190 ms) than in the experimental group (50 ms). Also, the three-way interaction of transition, CTI, and group was significant, F = 4.51, p = 0.045, η2 = 0.17. Switch costs were especially high in the control group in short CTI trials (250 ms), but could be reduced with a long CTI (130 m), which is a reduction of 48%. However, in the experimental group, switch costs were even more reduced with a long CTI (from 75 to 24 ms), which is a reduction of 68%. In addition, also the expected interaction of transition and CTI was significant, F = 28.29, p < 0.001, η2 = 0.563. Switch costs were higher in trials with a short CTI (162 ms) than with in trials with a long CTI (77 ms). As expected, the main effect of transition, F = 58.41, p < 0.001, η2 = 0.726, and of CTI were significant, too, F = 105.26, p < 0.001, η2 = 0.827. Neither the main effect of group was significant, F = 1.3, nor the interaction of CTI and group, F = 2.29.In the analysis of trials with a long preparation interval in the experimental group, we found small and non-significant switch-costs, like in the study of Ruge et al. . Hence, F = 1.2. Already in the first two blocks, the switch costs were smaller in the experimental group  than in the control group . The interaction of transition and group in the first two blocks was accordingly significant, F = 20.02, p < 0.001, η2 = 0.476.To check at which point of time the switch-costs decreased for the experimental group, we additionally took a look at the switch costs in the first eight blocks for the experimental and the control group. The main effect of group was not significant, F = 7.86, p = 0.01, η2 = 0.263. No other main effect or interaction was significant, Fs < 1.9.For the error rate, the four-way ANOVA for Block 8 and 9 was conducted. It revealed only a main effect of task transition with fewer errors in repetition trials than in switch trials (3.9 vs. 5.97%), F < 1, nor an interaction with switch costs.Also in the first eight blocks, no difference between group was shown, The aim of the present study was to investigate the role of action effects in task-switching. In order to examine this question, we designed a task-switching paradigm in which task-irrelevant, but predictable action-effects occurred. The task-response-effect mappings were practiced in eight acquisition blocks, so that action effects could be reliably anticipated. In a ninth (transfer) block, the action effects were random, so that the effects could no longer be anticipated in a task- and response-specific way. Additionally, the CTI was manipulated to examine effects of task preparation. The results showed that going from predictable  to unpredictable action-effects increased both RT and switch costs. However, this influence of anticipated action-effects was found only in trials with a short CTI.anticipated because otherwise an influence of stimuli that occur after response execution would be quite inexplicable. Hence, this influence of action effects is consistent with the already existing literature on effect anticipation . Although intended action-effects may be weighted more in the action code (Hommel, Our results also showed a difference in the switch costs between groups already in the acquisition phase. The experimental group with the predictable action-effects revealed smaller switch costs than the control group with the random action-effects, corroborating the results of the study of Ruge et al. . However Hommel, , having  Hommel, , 2011.This association seems to occur very fast, as the switch costs were already smaller in the first two blocks for the experimental group compared to the control group, although participants had only little opportunity to experience the predictable task-response-effect mapping. There is also evidence in the study of Kiesel and Hoffmann  that theTaken together, our data support the hypothesis that action effects play an important role in task implementation. Further, we can conclude that the learned associations are task-specific, that means task-response-effect associations instead of stimulus-response-effect associations, because the switch costs are affected. In this regard, one might ask if action effects do not only influence task-set activation but are even a part of the task set as a distinct task-set component. With respect to our study, this assumption is not yet necessary, but this may be a theoretical option that should be investigated in future studies. Either way, one should not forget that in our daily life, performing tasks has mostly external effects. Hence, to really understand how humans are dealing with multiple tasks in their daily life, action effects should be considered as an important influence in task performance in future studies.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


High-precision measurements implemented with light are desired in all fields of science. However, light acts as a wave, and the Rayleigh criterion in classical optics yields a diffraction limit that prevents obtaining a resolution smaller than the wavelength. Sub-wavelength interference has potential application in lithography because it beats the classical Rayleigh resolution limit. Here, we carefully study second-order correlation theory to establish the physics behind sub-wavelength interference in photon coincidence detection. A Young's double slit experiment with pseudo-thermal light is performed to test the second-order correlation pattern. The results show that when two point detectors are scanned in different ways, super sub-wavelength interference patterns can be obtained. We then provide a theoretical explanation for this surprising result, and demonstrate that this explanation is also suitable for the results found for entangled light. Furthermore, we discuss the limitations of these types of super sub-wavelength interference patterns in quantum lithography. This sub-wavelength interference pattern has been obtained using entangled light56λ/2. In our study, we find that using the appropriate scanning method, even super sub-wavelength interference patterns can be achieved. This surprising result encourages us to consider the physics behind this phenomenon and to determine whether we can achieve an interference pattern with arbitrarily high resolution. We address these questions in this manuscript and suggest how to obtain a full understanding of the second-order intensity correlation.Quantum lithography was first proposed by Boto E(−) and E(+) are operators for the negative and positive frequencies of the light field, ρ is the density matrix operator for the light source, and r is a transverse coordinate vector on the detection plane. The measured first-order correlation in Eq. (1) directly represents the distribution of light intensity on the detection plane.In interference and diffraction experiments with light, the observed quantity is the first-order correlation function of the light on a detection plane in the far-field region, In addition to the first-order correlation, light can have a second-order correlation, which is defined ash. If the source is thermal light, the second-order correlation function on the detection plane can be written as16In T(r′), the impulse response function can be expressed ask = 2π/λ is the wave number. If the light source is fully incoherent, as it is for thermal light, the first-order correlation function of light just emitted from the source δ(r) is the Dirac delta function, and n is the mean photon number. Here, we have assumed that the intensity distribution of the incoherent light on the source plane is uniformT2(r). This result shows that a well-defined Fourier-transform image of the transmittance of an object can be extracted from the second-order correlation function.Under the paraxial approximation and the assumption that the source is covered by an object with a transmission function x1, y1) and  of the detection plane. Simply but without loss of generality, a one-dimensional object such as a double-slit is usually employed to force the correlation function to become a two-variable function. For a one-dimensional symmetric double-slit placed along the y-axis, the correlation function is independent of the coordinates y1 and y2. In this way, the second-order correlation function in Eq. (5) can be further simplified to the formEq. (5) clearly shows that the second-order correlation function is a four-variable function of the transverse coordinates (λ = 457 nm and a rotating ground glass disk (GGD). The angular velocity of the GGD is maintained at ω = π rad/s. A double-slit is placed immediately after the GGD. The width of each slit is a = 0.038 mm, and the distance between centres of the two slits is d = 0.12 mm. The diffracted light is split into two parts after the double-slit using a beam splitter (BS) and is then recorded by two charge-coupled devices (CCDs). Both of the CCDs are located at z = 23 cm behind the double-slit; therefore the CCDs are placed in the Fraunhofer diffraction region with respect to the double-slit.The experimental setup is shown in 1 and CCD2. Here, we extract the relevant row data from the two CCDs, and construct the second-order correlation function. The result is shown in x1 and x2 are the horizontal coordinates on CCD1 and CCD2, respectively. Here, we use colour to represent the value of the second-order correlation function. This second-order correlation function is the two-dimensional second-order interference pattern of the double-slit.As shown above, the second-order correlation function Eq. (6) describes the correlation between two arbitrary points on CCD2 plane is fixed and another point detector on the CCD1 plane is moved along the x1 direction. The curve obtained along this line, is identical to the traditional Young's interference pattern, as shown in A cross-sectional curve with arbitrary spatial resolution can be obtained by choosing the correct line on the two-dimensional figure. Line (a) in x1 = x, and x2 = −2x, we obtain a narrower interference pattern than the sub-wavelength pattern, as shown in λ/3. If we set x1 = x, and x2 = x/2, as represented by line (d) in λ  in λ . The expet al.et al.et al. found that the two-photon double-slit interference pattern is 2-fold narrower than the traditional pattern obtained using coherent light. If one assumes that the entangled photon pairs are generated simultaneously at either the upper or lower-slit and propagate to the detection plane independently, the coincidence count at the detection plane can be approximately expressed in the form262728The above conclusion can be applied to sub-wavelength interference from an entangled light source. In this case, Angelo et al.For simplicity, the single-slit diffraction function has been ignored. As shown in Eq. (7), the second-order correlation function is a two-dimensional function of the detection plane coordinates. In x1 = x, x2 = 2x) and , respectively. In λ/3 and 3λ, respectively. As described in Ref. As in the analysis for thermal light, line (a) in Based on the above results for both thermal and entangled light, it is clear that a cross-sectional curve can be constructed with arbitrary spatial resolution by choosing the scanning method for the detectors in the detection plane. However, cross-sectional curves represent only part of the information obtained from the second-order interference. To obtain all of the information, we should consider all possible combinations of arguments for the second-order correlation function. For example, when including all possible variations of the two arguments, the full picture of the second-order interference of a double-slit is a two-dimensional pattern.N-photon absorption) materials, although proof-of-principle experiments that display certain aspects of quantum lithography have been performed. In these proof-of-principle experiments, N photons should appear at same point on the N detectors simultaneously because the N-photon absorber is simulated by N-fold coincidence. From this perspective, we can see that if N detectors scan with different paces and in different directions, it is impossible to take advantage of the resulting super sub-wavelength patterns to achieve quantum lithography. In other words, for thermal light, the second-order interference pattern becomes flat when two detectors scan in the same directions (x1 = x2), and this method clearly can-not be used for writing sub-Rayleigh structures. However, for entangled light, a sub-wavelength interference pattern is obtained when two detectors are scanned with x1 = x2 , represent the positions of two detectors. Thus, all the information available from the second-order correlation function can be obtained only by considering all possible combinations of x1 and x2 in the detection plane. In previous sub-wavelength interference experiments using a thermal light source5λ/2 spatial resolution cross-sectional curves were obtained when two point detectors were scanned in opposite directions . However, there are no mathematical or physical reasons to limit the scanning methods for two detectors to the coordinates x1 and x2 in Eq. (6). For example, if the two point detectors move in the detection plane along x1 = x, x2 = x/2, one can obtain the cross-sectional curve with spatial resolution 2λ. In contrast, if the two point detectors move in the detection plane along x1 = x, x2 = −2x, one can obtain cross-sectional curve with spatial resolution λ/3. In general, the cross-sectional curve with spatial resolution Nλ or N > 0) can be obtained when the two point detectors scan in the detection plane along x1 = x, x1 = x, x2 = (1 − N)x, respectively. Therefore a cross-sectional curve with arbitrary spatial resolution can be obtained merely by choosing an appropriate scanning method. The result that the physical quantity G(2) is not independent of the scanning method appears surprising. However, noting the second-order correlation function Eq. (6), we can find the reason for this effect—the second-order correlation pattern is a two-dimensional surface rather than a one-dimensional curve. To further clarify our argument, we perform a second-order interference experiment using pseudo-thermal light with a setup similar to that employed in Ref. The two variables d in Eq. , represed in Eq. , represeR.L. constructed and operated the experiment, and collected the data. P.Z. and F.L. devised and designed the experiment. Y.Z. and H.G. performed the theoretical calculations. All authors contributed to the manuscript.


Estimating π1 is usually done through P-values, but computing P-values can be viewed as a nuisance and potentially problematic step. We evaluated methods for estimating π1 directly from test statistics, circumventing the need to compute P-values. We adapted existing methodology for estimating π1 from t- and z-statistics so that π1 could be estimated from other statistics. We compared the quality of these estimates to estimates generated by two established methods for estimating π1 from P-values. Overall, methods varied widely in bias and variability. The least biased and least variable estimates of π1, the proportion of differentially expressed genes, were produced by applying the “convest” mixture model method to P-values computed from a pooled permutation null distribution. Estimates computed directly from test statistics rather than P-values did not reliably perform well.The goal of many microarray studies is to identify genes that are differentially expressed between two classes or populations. Many data analysts choose to estimate the false discovery rate (FDR) associated with the list of genes declared differentially expressed. Estimating an FDR largely reduces to estimating  Gene expression microarrays are a standard tool for large-scale measurement of gene expression. Microarrays are widely used to detect genes that are differentially expressed (DE) across different groups. Methodology for detecting DE genes has matured over the past decade. Methods have evolved from simple fold-change rules, to the use of classical statistical methods, to the use of test statistics developed specifically for the microarray context . The search for DE genes is usually done in the framework of statistical hypothesis testing. A hypothesis test is performed for each gene. Since microarray studies usually involve tens of thousands of genes, detecting DE genes automatically involves multiple testing issues. Rather than controlling false positives through the traditional family-wise error rate (FWER), most researchers prefer to consider the false discovery rate (FDR). The false discovery rate is an alternative to FWER that was originally proposed by Benjamini and Hochberg . The FDRP-value for every gene. (3) For some threshold α close to 0 , call all genes with P-values less than α significant. (4) Estimate the FDR associated with the list of significant genes. In reality steps (3) and (4) might be done iteratively, with the P-value threshold adjusted depending on the estimated FDR resulting from step (4). Next, we review methodology for these 4 steps, which will introduce the question this paper will address.Controlling the FWER is too conservative in the microarray setting, because usually investigators are willing to get a small proportion of false positives in exchange for a sizeable list of potentially DE genes for further study. There is a near-consensus that FDR-estimation procedures are the preferred method for addressing multiple testing in the microarray context , 3. A tyXjk be the (possibly log-transformed) level of expression of a given gene in sample j of population k. Let nk be the number of samples drawn from population k. Define k. Also define k. Typically expression levels are compared across two populations or groups . To identify DE genes, one could use the classical two-sample t-statistic:t-statistic relies on estimates of the expression variances. With the sample sizes typical of most microarray studies (only a few samples per group), these estimates are very unstable. Given the large number of genes in microarray studies, some genes will exhibit a low variance by chance. In real data one often finds that the genes with the largest t-statistics are those with the smallest denominators, not necessarily those that are DE between groups. Step (1) involves the choice of test statistic. The earliest approaches to identifying DE genes were simple fold-change rules. An example of a fold-change rule is to declare a gene DE if its average expression level varies by more than a factor of two between the comparison groups. However, such rules are generally considered unsatisfactory because they do not incorporate the variability of the data, and there is no associated level of confidence in the conclusion of declaring a gene DE . It was s-statistic,s-statistic is identical to the t-statistic with the addition of a constant, δ, to the denominator. The constant δ has the effect of stabilizing the denominator of the t-statistic. We will refer to δ as the stabilizing constant.Tusher et al.  proposedδ. Tusher et al. [δ to ensure that the variance of the s-statistic “is independent of gene expression.” Broberg [t-statistic denominators as δ, and Xie et al. [δ: 0; 5th, 50th, and 90th percentiles, and the limit as it approached infinity and found the 90th percentile worked best. There is no consensus on the best way to choose r et al.  state th Broberg  used thee et al.  used thee et al.  comparedt-statistic for detecting DE genes. In each of the papers mentioned above , sim, simt-stn et al.  assessed and Cao  employedP-value computed for each gene to test the null hypothesis that the gene is not DE. MMMs consider these P-values as a mixture of P-values for genes for which the null hypothesis is true and P-values for which the null hypothesis is false. Estimators of the FDR have the formα is the P-value threshold, π0 is the proportion of non-DE genes, π1 = (1 − π0) is the proportion of DE genes, and Fp(·) is the cumulative distribution function of all P-values. Most MMMs use the observed number of P-values less than α to estimate Fp(α). Therefore, most MMMs differ only in their estimates of π1. Step (4) involves estimating the FDR associated with a list of genes declared significant. Many different “Mixture Model Methods” (MMMs)  have beeP-value—step (2)—one needs to know the distribution of the test statistic under the null hypothesis. Obtaining an empirical null distribution by permutation is a very popular choice in the microarray context. However, Kerr [P-values resulting from permutation tests and MMMs may be incompatible since permutation-test P-values may not satisfy all of the assumptions implicit in MMM methodology. Therefore, estimating the FDR directly from test statistics—skipping step (2) altogether—could be advantageous. In order to compute a er, Kerr  showed tThere are few tools available to estimate the FDR directly from test statistics. One such tool, “locfdr” , operateP-value together with a MMM for P-values. In this approach, a single null distribution of the test statistic is estimated by pooling all the permutation null test statistics across genes. Kerr [P-values” are different from permutation test P-values. However, “pooled null P-values” have the attractive feature that they have a monotone relationship with the test statistic. For our second class of methods for estimating π1, we computed pooled null permutation P-values and then estimated the FDR with an MMM. We used two MMMs that we have seen to work well: “qvalue” [There are many different Specialized Test Statistics to choose from; we use the SAM-statistic in our investigation due to its simplicity and popularity. We compare results to a procedure in current common practice, which is to use a variant of a permutation-test es. Kerr  showed t“qvalue”  and “con“qvalue” .s-statistics or any other Specialized Test Statistics as input. Since “fdrtool” accepts t-statistics, and s-statistics are similar to t-statistics, we investigated whether we could apply “fdrtool” to s-statistics and get accurate results. Our idea was to compute both s-statistics and t-statistics on the same data, and then rescale the s-statistics to have the same spread as the corresponding t-statistics. We considered two measures of variability: standard deviation and interquartile range. In addition, we considered four different s-statistics, each using a different stabilizing constant term in the denominator. We designed simulations to investigate the accuracy of using “fdrtool” to estimate FDR's from s-statistics in this way. Our simulated data were based on real microarray data.The methodology of “fdrtool” is not designed to take t-statistic and an s-statistic for each gene. Due to the addition of the stabilizing constant in the denominator of the s-statistic, the 100 s-statistic values will be closer to zero than the 100 t-statistic values. In other words, the addition of δ to the denominator pulls the s-statistic values toward zero, so that the variance of the s-statistics across genes is smaller than the variance of the t-statistics. As a simple example, suppose that we have 100 genes with measured expression levels on two samples from two groups we wish to compare. We compute a s-statistics. We examined two measures of variability: the standard deviation (SD) and the interquartile range (IQR). We took the ratio of the variability of the t-statistic to the variability of the s-statistic as our rescaling factor. For the SD, we transformed the s-statistics by multiplying them by the factor (SDT/SDS), where SDT and SDS are the standard deviations across genes of t-statistics and s-statistics, respectively. The rescaling factor for the IQR was similarly defined, with IQR in the place of SD. Note that rescaling the s-statistics maintains their rank order, which retains the s-statistic's advantages for accurately detecting DE genes.Accordingly, we used a measure of the difference in variability of the statistics across genes as the rescaling factor to rescale the s-statistics one must choose the value of the stabilizing constant δ in the s-statistic denominator. We considered four choices for δ, and so defined four different s-statistics: the s30, s50, s70, and s90. The s30-statistic uses the 30th percentile of all t-statistic denominators as δ, the s50-statistic uses the 50th percentile of all t-statistic denominators as δ, and so forth. In summary, for each simulated data set, we computed the t-statistic and the four s-statistics. We transformed the s-statistics by two different rescaling factors (SD and IQR). When loading the rescaled s-statistics into “fdrtool,” we evaluated two different specifications to the software. We could specify that the input was t-statistics or z-statistics. We evaluated the performance of these five statistics using the four different combinations of rescaling factors and two different input options.In computing The design of the simulation study is that same as Kerr (2009) , and we For each gene, we calculated the sample mean and sample standard deviation of that gene in each population group (CEU and CHB). We rounded the sample mean values to the nearest tenth digit, so that means could be unambiguously declared equal or unequal between groups. We simulated CEU and CHB sample data  from independent normal distributions with parameters based on the values from the actual data. We simulated datasets of 10,000 transcripts.π1) were differentially expressed in the mean. For the simulated DE genes, the difference in means for the simulated CEU and CHB samples was taken from the observed sample means in the real data. The variances of the simulated CEU and CHB samples differed between the three simulation types.We ran three types of simulations: EV, UV1, and UV2. In each simulation, a proportion of genes  simulations, the standard deviation of both the simulated CEU sample and the simulated CHB sample came from the observed standard deviation from the CEU data. This was not the case in the UV  simulations. In the UV1 simulations, the standard deviation of the simulated CEU sample came from the observed standard deviation from the CEU data, and the standard deviation of the simulated CHB sample came from the observed standard deviation from the CHB data. In the UV2 simulations, this was reversed: the standard deviation of the simulated CEU sample came from the observed standard deviation from the CHB data, and the standard deviation of the simulated CHB sample came from the observed standard deviation from the CEU data.π1: 0.01, 0.05, 0.10, and 0.25, and three different sample sizes: large, intermediate, and small. Letting nCEU and nCHB denote the sample sizes for CEU and CHB, respectively, our sample sizes were as follows: large  = , intermediate  = , and small  = . Note that all of the sample sizes maintain the 4 : 3 ratio of the original data. With three sample sizes, four values of  π1, three simulation types, two rescaling factors (SD and IQR), and two input specification options , there were a total of 144 different simulation scenarios. After examining the results, we performed additional EV simulations for π1 = 0.005, 0.02, 0.03, and 0.04 for the three sample sizes, two rescaling factors, and two input specification options, adding an additional 48 simulation scenarios, for a total of 192 simulation scenarios. We repeated each scenario 20 times. We initially simulated data for four different values of s-statistics outperformed the t-statistic in identifying DE genes. s-statistics outperform the t-statistic across all sample sizes and values of π1. This difference in performance is more marked for the smaller sample sizes than the larger sample sizes, across all values of π1. The four s-statistics are generally close in performance. The s30-, s50-, and s70-statistics perform comparably. The s90-statistic (purple curve) stands slightly apart from the other three, giving less sensitivity at low false positive rates and better sensitivity at higher false positive rates. The difference in the ROC curves for the s90-statistic and the other s-statistics is more pronounced in the smaller sample sizes. However, the difference between the s90-statistic and the other s-statistics is not as large as the difference between the t-statistic and the four s-statistics.First, we verified that the s-statistics when “fdrtool” treated them as either t-statistics or z-statistics. Specifying in “fdrtool” that the inputted s-statistics were t-statistics . In the first row of π1 have a modest conservative bias for the s30- and s50-statistics in the two smaller sample sizes. However, for higher proportions of DE genes , the “fdrtool” estimates of π1 are poor, with excessive conservative bias. This result held regardless of whether statistics were rescaled with IQR , but showed anticonservative bias for low percentages of DE genes (π1 = 0.01). Bias and variability is mostly improved over π1 = 0.01. All results presented here are for the EV simulations; results for the UV1 and UV2 simulations were similar (see supplementary File 1).In contrast to the results for the atistics  worked bP-value. In this approach, a single null distribution of the test statistic is estimated by pooling all the permutation null test statistics across genes. “Pooled null P-values” can be computed from a single empirical null distribution. We also obtained estimates of π1 using pooled null P-values and then estimating the FDR with an MMM. We obtained estimates of π1 using two MMMs that we have seen to work well: “qvalue” [A popular approach in practice is to compute a variant of a permutation-test “qvalue”  and “con“qvalue” .qvalue” results for the EV simulations. For a low percentage of DE genes , the “qvalue” results , the story is similar, with “qvalue” estimates generally showing less bias than “fdrtool” estimates, but greater variability. However, the higher variability in the “qvalue” results seems more clearly acceptable when π1 = 0.10, 0.25, given the dramatic reduction in bias. Interestingly, “qvalue” does not appear to perform worse for the t-statistic than it does for the s-statistics, whereas “fdrtool” almost always performs better for s-statistics. The comparison of the “fdrtool” results to the “qvalue” results was similar for the UV1 and UV2 simulations .π1 for the EV simulations. Overall, the “convest” estimates show less bias and comparable or less variability than the “fdrtool” or “qvalue” results. Thus, although it relies on P-values that could be considered invalid as permutation test P-values [π1 (and hence FDR) in terms of bias and variability. Similar to the “qvalue” results, “convest” performs comparably for t-statistics as for s-statistics.P-values , “convesP-values and was not prohibitive. The more important difference is that “convest” requires P-values, which we computed via permutation, whereas the “fdrtool” is applied directly to test statistics and does not require any data permutations.The “convest” method took longer to compute than the “fdrtool” approach (approximately 20 times as long). However, computation time was on the order of 6.5 seconds for 1 set of 10,000 An important limitation of this study is that all simulations were based on one dataset. We also did not explore different correlation structures for the simulated gene expression.π1, the proportion of differentially expressed genes, from microarray data. The approaches were in two classes: (1) adapting the methodology of “fdrtool” to Specialized Test Statistics and (2) applying mixture model methods (MMMs) to P-values computed from a pooled permutation null distribution. The best-performing method was in the second class, using the MMM “convest” to P-values computed from a pooled permutation null distribution. Overall, estimates of π1 exhibited the least bias and variability, and bias tended to be conservative rather than anti-conservative.We compared approaches for estimating π1 adapted existing methodology of “fdrtool” to s-statistics. Interestingly, “fdrtool” generally performed better for s-statistics than it did for t-statistics, even though the empirical modeling is designed for t-statistics. The performance of the “fdrtool”-based approach varied substantially on the proportion of differentially expressed gene. However, the “convest”-based approach outperformed the fdrtool approach in almost all scenarios.The first class of approaches for estimating The Supplementary Material File contains the complete set of results for all simulation scenarios examined in the research.Click here for additional data file.


It has been proposed that, contrary to the “classical” view in which cortical action potentials are encoded as purely digital, all-or-none events, action potential (AP) shape may be used by the brain for representing and processing information -4. Corti