We develop a heuristic called DOCKS to find a compact UHS, which works in two phases: The first phase is solved optimally, and for the second we propose several efficient heuristics, trading set size for speed and memory. The use of heuristics is motivated by showing the NP-hardness of a closely related problem. We show that DOCKS works well in practice and produces UHSs that are very close to a theoretical lower bound. We present results for various values of k and L and by applying them to real genomes show that UHSs indeed improve over minimizers. In particular, DOCKS uses less than 30% of the 10-mers needed to span the human genome compared to minimizers. The software and computed UHSs are freely available at github.com/Shamir-Lab/DOCKS/ and acgt.cs.tau.ac.il/docks/, respectively.With the rapidly increasing volume of deep sequencing data, more efficient algorithms and data structures are needed. Minimizers are a central recent paradigm that has improved various sequence analysis tasks, including hashing for faster read overlap detection, sparse suffix arrays for creating smaller indexes, and Bloom filters for speeding up sequence search. Here, we propose an alternative paradigm that can lead to substantial further improvement in these and other tasks. For integers  k-mer sets (DNA words of length k) to represent the sequences in a dataset. The runtime and memory usage of these highly depend on the size of the k-mer sets used. Thus, a minimum-size k-mer hitting set, namely, a set of k-mers that hit (have non-empty overlap with) all sequences, is desirable. In this work, we create universal k-mer hitting sets that hit any L-long sequence. We present several heuristic approaches for constructing such small sets; the approaches vary in the trade-off between the size of the produced set and runtime and memory usage. We show the benefit in practice of using the produced universal k-mer hitting sets compared to minimizers and randomly created hitting sets on the human genome.High-throughput sequencing data has been accumulating at an extreme pace. The need to efficiently analyze and process it has become a critical challenge of the field. Many of the data structures and algorithms for this task rely on  The pace of high-throughput sequencing keeps accelerating as it becomes cheaper and faster and with it the need for faster and more memory efficient genomic analysis methods grows. The NIH Sequence Read Archive, for example, currently contains over 12 petabases of sequence data and is growing at a fast pace. Increased use of sequence-based assays  in research and in clinical settings creates high computational processing burdens. Metagenomic studies generate even larger sequencing datasets. New fundamental computational ideas are essential to manage and analyze these data.L, its minimizer is the lexicographically smallest k-mer in it [S of any length its minimizer set is the set of minimizers of every L-long subsequence in S. Hence, every window of length L in S is represented in the set, and a minimizer set for a sequence S constitutes a succinct representation for it. As we discuss below, minimizers have had numerous applications in sequence analysis.The minimizer approach has been extremely successful in increasing the efficiency of several sequence analysis challenges. Given a sequence of length er in it , 2. For S, we introduce the notion of a universal hitting set. For integers k, L, a set Uk,L is called a universal hitting set of k-mers (UHS) if every possible sequence of length L must contain at least one k-mer from Uk,L. The set of all k-mers is a trivial UHS, but it does not provide any useful reduction in computational resources needed. Hence, our main computational problem is:Here, we generalize and improve on the minimizer idea. To avoid dependence on a particular sequence Problem 1. Given k and L, find a smallest universal hitting set of k-mers.A small UHS has a variety of applications in speeding up genomic analyses since it can be used where minimizers have been used in the past. For example:Hashing for read overlapping. A naïve read overlapper must test O(n2) pairs of reads to see whether they overlap (where n is the number of reads). If we require an overlap of length L, any pair of reads with such an overlap must share a k-mer from set Uk,L in this overlapped region. By bucketing reads into bins according to the universal k-mers they contain, we need only test pairs of reads in the same bucket. The number of buckets is limited by |Uk,L|.Sparse suffix arrays. A sparse suffix array of a string S saves memory by storing an index for only every sth position in S [q, we perform at most s queries starting from indices 0, …, s − 1 in q; one of these queries will intersect a position stored in the suffix array. Using Uk,L, we can instead store only positions in S that start with a k-mer in Uk,L. Any query with |q| ≥ L must contain one of these selected k-mers and will be matched when searching the suffix array. This approach has been applied with minimizers [ion in S . To quernimizers  to good Bloom filters to speed up sequence search. Bloom filters have been used to speed up sequence search by storing k-mers present in a read set for quick testing [k-mers present in a read set are stored in these filters. If, instead, only the set of k-mers in Uk,L is stored, any window of length ≥ L is still guaranteed to contain one of these representative queries, potentially reducing the size of Bloom filters that must be maintained. testing , 6. In cet al. [et al. [k-mers by hashing them to their 4-mer minimizer to efficiently construct a de Bruijn graph for assembly. SparseAssembler [g-th k-mer in the sequence . Kraken [k-mers during metagenome sequence classification. KMC 2 [k-mer occurrences. The Locally Consistent Parsing (LCP) [Minimizers have been used for some of these and similar applications , 7–9. Thet al.  for geno [et al. . For exa [et al.  compressssembler  represen. Kraken  uses minn. KMC 2  uses minng (LCP)  algorithng (LCP)  uses corA small UHS, if it can be found, has a number of advantages over minimizers for these applications:k-mers , whereas we show that we can often generate UHSs smaller by a factor of nearly k. We also demonstrate on real genomic sequences that the number of UHS k-mers needed to process them is substantially smaller.The set of minimizers for a given collection of reads may be as dense as the complete set of k and L, a set of universal k-mers needs to be computed only once and not recomputed for every dataset.For any k-mers if they are sampled according to a UHS. This will enable easier comparison and integration of the datasets.The hash buckets, sparse suffix arrays, and Bloom filters created for different datasets will contain a comparable set of k-mers to use.One does not need to look at the reads or to build a dataset-specific de Bruijn graph in order to decide which k (see Definition 1 below). This is the viewpoint we take for most of this study:Problem 1 can be rephrased as a problem on the complete de Bruijn graph of order Problem 2. Given a de Bruijn graph Dk of order k and an integer L, find a smallest set of vertices Uk,L such that any path in Dk of length ℓ = L − k passes through at least one vertex of Uk,L.edges in it. We show that the related problem of finding a minimum-size k-mer set that hits every string in a given set L-long strings is NP-hard. This problem differs from ours, in that the set k-mers that hits every sequence in a particular data set is hard further motivates the need for a universal set that can be computed once for any input sequence. Our main contribution is an algorithm called DOCKS that finds a compact set of k-mers that hits any L-long sequence. We also provide several variants of the algorithm, trading-off some solution quality for speed. We show empirically that the produced sets are often close to a theoretical lower bound, implying their near-optimality. Our use of a greedy heuristic is motivated by the fact that finding a minimum-size ℓ-long path cover in a graph G is NP-hard when G is a directed acyclic graph (DAG). We report on the size of the universal k-mer hitting set produced by DOCKS and demonstrate on genomic datasets that we can more uniformly cover sequences with a smaller set of k-mers than is possible using minimizers. For example, we show that the number of k-mers needed to cover the human genome using a UHS is less than one third of that required by minimizers.Here and throughout, the length of a path is the number of github.com/Shamir-Lab/DOCKS/. Universal sets of k-mers computed by DOCKS for a range of values of L and k are freely available at acgt.cs.tau.ac.il/docks/. A preliminary version of this study appeared in [The software to compute small UHSs is freely available at eared in .k denotes the length of a k-mer word, while L denotes the length of the long sequences.Throughout this paper, Definition 1 (de Bruijn Graph). A de Bruijn graph of order k over alphabet Σ is a directed graph in which every vertex has an associated label (a string over Σ) of length k (k-mer) and every edge has an associated label of length k + 1. There are exactly |Σ|k vertices in a de Bruijn graph, each representing a unique k-mer. If an edge  has label l, then the label of u must be the k-prefix (prefix of length k) of l and the label of v must be the k-suffix (suffix of length k) of l. A complete de Bruijn graph contains all possible edges of this type, which represent together all (k + 1)-mers over Σ.v0, e0, v1, e1, v2, …, vn of length n spells a sequence s of length n + k such that the label of vi occurs in s starting at position i for all 0 ≤ i ≤ n, and the label of ei occurs in s starting at position i for all 0 ≤ i ≤ n − 1. Note that vertices and edges may repeat in a path.Every path in a de Bruijn graph represents a sequence. A path k-mers intersecting sequences over an alphabet Σ:We define terminology for Definition 2 (hits). We say that k-mer whits string S, denoted w ⊆ S, if w appears as a contiguous substring in S. k-mer set Xhits string S if there exists w ∈ X s.t. w ⊆ S. Define hit = {S ∈ ΣL ∣ w ⊆ S} for k-mer w and length L, where ΣL is the set of all L-long substrings over alphabet Σ. Define k-mers from Problem 1 is then a set Uk,L which satisfies hit = ΣL.The universal set of hitting k, L)-hitting set. As we prove in the Appendix, the problem of finding a minimum  k-mer set that hits a given set of input sequences is NP-hard -HITTING SET in k-mer set. This algorithm works on the de Bruijn graph of order k in two steps: first it finds and removes a minimum-size k-mer set hitting all infinite sequences, and then it finds and removes additional k-mers in order to hit all remaining L-long sequences. We now describe these two steps in detail.It is not known how to efficiently find a minimum universal ), where M(k) is the size of the minimum unavoidable set. M(k) converges to |Σ|k/k (an exact formula is given in O(|Σ|k).The problem of finding a minimum-size t length . Note thput size . The oriput size . Its runk is equivalent to a set of vertices in a complete de Bruijn graph of order k whose removal turns it into a directed acyclic graph (DAG). Each k-mer in the set corresponds to a vertex, and the removal of vertices from every cycle guarantees that no infinite sequence is represented as a path in the graph. This set is known as a decycling set.An unavoidable set of constant length L-long sequences that avoid that set. Thus, we need additional k-mers to hit those. If we consider the graph formulation, after removal of a decycling set from the graph we are left with a DAG, which may contain (L − k)-long paths representing L-long sequences. We need to remove additional vertices, so that there is no path of length ℓ = L − k. The problem of finding a minimum-size set of vertices that hit all ℓ-long paths in a general directed acyclic graph is known to be NP-hard, as we review in the Appendix  of a vertex v to be the number of paths of length ℓ that contain v. The main observation is that we can calculate the hitting number of each vertex efficiently using dynamic programming. The solution is based on calculating the number of paths of length i that terminate at vertex v, and the number of paths of length i that start at vertex v, for all v ∈ V and 0 ≤ i ≤ ℓ. Then, the number of ℓ-long paths through v is directly computable from these values by breaking any path into an i-long path ending at v and an (ℓ − i)-long path starting at v, for all possible values of i. We set ℓ = L − k to get the desired hitting number of each vertex.Our initial algorithm is based on the greedy algorithm for the minimum hitting set . We defiG′ =  be the directed acyclic graph, after removal of the decycling set. Denote by D and F matrices of size |V′| × (ℓ + 1) where D is the number of i-long paths in G′ starting at vertex v and F is the number of i-long paths ending at vertex v.Specifically, let D and F is done recursively as follows:ℓ-long paths that vertex v participates in, we sum:E′|)) times ℓ, giving a running time of O(|Σ|k+1 ⋅ ℓ) for ℓ = L − k.The calculation of k and remove it from the graph, obtaining a DAG. Then, we repeatedly remove a vertex v with the largest hitting number T until there are no ℓ-long paths, recomputing T for all remaining vertices u after each removal. This is summarized below (Algorithm 1).The full algorithm combines the two steps. First, we find a decycling set in a complete de Bruijn graph of order G of order k, set ℓ = L − k.1: Generate a complete de Bruijn graph X using Mykkeltveit’s algorithm.2: Find a decycling vertex set X from graph G, resulting in G′.3: Remove all vertices in while there are still paths of length ℓdo4: D and F for each vertex v and 0 ≤ i ≤ ℓ.5:  Calculate T for each vertex v.6:  Calculate G′, and add it to set X.7:  Remove a vertex with maximum hitting number from end while8: X.9: Output set O(|Σ|k). In the second phase, each iteration calculates the hitting number of all vertices in time O(|Σ|k+1ℓ). The number of iterations is 1 + p, where p is the number of vertices removed. Thus, the total running time is dominated by steps 4–8 and is O((1 + p)|Σ|k+1ℓ).Finding the decycling set takes k limits the range of k to which it can be applied , can be calculated more quickly and serve as an estimate of T. We call this heuristic DOCKSany (Algorithm 2).In order to extend the range of v with maximum T(v) in each iteration. To compute T(v) for all v, the vertices in the current graph G′ =  are first sorted in topological order v1 ≤ … ≤ vn. Define F(v) as the number of paths ending at v. The vertices are visited in topological order and the incoming edges into v are used to compute:D(v), the number of paths starting at v is computed by visiting the vertices in reverse topological order and computing.T(v) is then calculated for all vertices as:DOCKSany has the same structure as DOCKS, but with one difference: it removes a node v with maximum T(v) is removed, G′ is updated, and the process is repeated until there are no paths of length ℓ in the graph.A vertex G of order k, set ℓ = L − k.1: Generate a complete de Bruijn graph X using Mykkeltveit’s algorithm.2: Find a decycling vertex set X from graph G, resulting in G′.3: Remove all vertices in while there are still paths of length ℓdo4: D(v) and F(v) at each vertex v.5:  Calculate T(v) of paths passing through each vertex v.6:  Calculate the number v with maximum T(v) from G′, and add it to set X.7:  Remove a vertex end while8: X.9: Output set D(v) for all v requires visiting each edge in the graph once, and hence takes O(|Σ|k+1). The time for computing F(v) for all v is the same. Hence, T is computable in O(|Σ|k+1) time. Computing the longest path in a DAG (step 4) also requires O(|Σ|k+1). If p vertices are removed, then the total runtime for this algorithm is O((1 + p)|Σ|k+1), a factor of Θ(ℓ) faster than the DOCKS algorithm. The space complexity is also smaller, O(|Σ|k+1) vs. O(ℓ|Σ|k+1) for DOCKS.Computing L. The value of L only determines when the algorithm terminates. Thus, hitting sets for all values of L or larger can be computed in one run. This is in contrast with DOCKS, in which the hitting number of each vertex depends on L, and so DOCKS must be run for each desired value of L.In addition to shorter runtimes and decreased memory usage, this heuristic offers one more advantage over the original DOCKS algorithm. The vertex removal choice is independent of k, we can further speed up DOCKSany as follows. In the DOCKSanyX heuristic, the top X vertices, ranked by the hitting number T(v), are removed (in step 7) in each iteration. This can shorten the running time of each iteration by a factor of X, but may produce larger hitting set solutions.Finally, in order to calculate the hitting set for even larger k-mer hitting set as an integer linear program (ILP). In the ILP formulation there are |Σ|k binary variables xi representing whether vertex i is in the solution hitting set. There are also |Σ|k variables Li representing an upper bound on the number of edges in the longest path ending at vertex i. The constraints on Li guarantee that the vertices chosen remove all ℓ-long paths (ℓ = L − k) from the graph. The ILP is defined as follows:To investigate whether optimal solutions can be found practically, we formulate the problem of the minimal universal E contains all |Σ|k+1 possible edges. The constraint on edge  requires that if v is not in the set then Lv ≥ 1 + Lu. The validity of this formulation is proven in the Appendix -mer hitting set, denote by S an (L + j)-long sequence. By definition, there must be at least one k-mer w ∈ Uk, L that hits S’s L-long prefix. Uk+j, L+j contains all (k + j)-mers w ⋅ x, where x is any j-mer. Thus, it must contain a (k + j)-mer that hits S.The DOCKS variants described above have exponential dependence in U10,20 we obtain U20,30. The size of the set U10,20 is |U10,20| = c ⋅ dec10, where k = 10 . Consequently, a minimum hitting set Uk, L has a size ≥ C ≥ |Σ|k/k.A decycling set necessarily contains a keltveit , states Lmax, the length of the longest sequence in a complete de Bruijn graph after a minimum decycling set computed using Mykkeltveit’s algorithm is removed, for k = 2 to 14. For this range of k, the length of sequences avoiding the decycling set can theoretically be appropriate for long-read sequencing technologies, such as PacBio [L > 1000. Such long reads are all hit by a decycling set according to k ≤ 14 . However, many short reads can avoid the decycling set. Additional k-mers must be selected to obtain a hitting set for shorter sequences. Note that different minimum decycling sets may result in different lengths of the longest path in the remaining DAG. Mykkeltveit’s approach is different from that of Champarnaud et al. (2004), and the former has an advantage in producing solutions with shorter longest paths [s PacBio  and Nanos PacBio , which pst paths .k and L: 5 ≤ k ≤ 10 and 20 ≤ L ≤ 200, in increments of 10. The values of k are typical lengths for minimizers, and the L values are typical lengths of short reads. Note that in some applications, like KMC 2 [k there) corresponds to our L parameter, and the length of the minimizers (m in KMC 2) corresponds to our k parameter.We implemented and ran DOCKS over a range of ke KMC 2  and Krakke KMC 2 , the lenk-mers included in the solution set decreases with L. It is easier to hit longer sequences as they contain more k-mers. In addition, running times and memory usage increase exponentially with k. For k = 10, DOCKS terminated after more than 2.5 hours and used more than 1 GB of memory. For k = 11 and L = 20 running time was 128 hours. Hence, DOCKS runtime would be prohibitively long for larger values of k. Running times were benchmarked on a single CPU of a 20-CPU Intel Xeon E5-2650 (2.3GHz) machine with 384GB 2133MHz RAM.The results are summarized in k. For k = 10 and L = 20 the number of added k-mers roughly equals the size of the decycling set, while for k = 5 and L = 20 it is only 20% larger. For all values of k, the ratio improves as L grows. We also compared DOCKS to a pure greedy algorithm that repeatedly removes a vertex with a maximum hitting number, without removing a decycling set first. For almost all combinations of  the size of the produced set, runtime and memory of the greedy algorithm were far greater than those of DOCKS , and it increased with L, as opposed to DOCKS’s runtime, which decreased with L.k ≤ 11 and 20 ≤ L ≤ 200. The results for k = 10 are shown in L, and that gap grows with k: from 10% for k = 5 to 60% larger for k = 10. Set sizes of DOCKS and DOCKSany are closer as L increases and both approach the size of the decycling set. In terms of running time, on the other hand, we see a great benefit in using DOCKSany as runtimes decrease to a small fraction of the DOCKS running times for the larger values of k. We also see reduced memory usage for larger values of k and L , so we turn to another heuristic to increase runtime on the expense of larger set sizes.We ran DOCKSany for 5 ≤ k = 10, 20 ≤ L ≤ 200 and X = 5i for 0 ≤ i ≤ 5 . Remarkably, for k = 10, the size of the solution is similar to that of DOCKSany while there is a factor of > 100 × speedup. The results, runtime and memory usage of DOCKSanyX are in We tested the performance of DOCKSanyX for Fig D in . As expeally see . Fig 2 ck ≤ 10 with 20 ≤ L ≤ 200. To save time, we set the starting feasible solution to be the DOCKS solution. We let the solver run for up to one day for each k and L. This did not necessarily produce an optimal solution to the ILP, although the solver was often able to improve on the starting DOCKS solution. In k. Improvements diminish as L increases, since the set sizes approach the theoretical lower bound, i.e., the size of the minimum decycling set. Letting the ILP solver run for longer times may provide further improvements for small values of L.We solved the ILP using Gurobi 6.5.2  for 5 ≤ k-mer in each window of w consecutive k-mers in order to reduce storage size for sequence comparison. We can improve the minimizers algorithm by choosing the lexicographically smallest k-mer that is in the DOCKS set for the corresponding k and L parameters (i.e. L = k + w − 1). Such a k-mer is guaranteed to exist, as by construction, every window of length w contains a k-mer in the UHS. We ran the minimizer selection algorithm and DOCKS-based selection on four different genomes, using k = 10 and L = 30: the entire human reference genome (GRCh38), the bacteria A. tropicalis strain NBRC 16470, and C. crescentus strain CB15, the worm C. elegans assembly WBcel235. For comparison, we also included the results when using the minimizer according to a random ordering of the k-mers, instead of lexicographic. This random ordering typically improves over minimizers since it avoids the problem of always selecting the common poly-A homopolymer.The minimizer algorithm  selects k-mers and those k-mers are more widely spread apart in the sequence. The advantage of DOCKS grows as the sequence length increases, having a size ≈ 85% of the next-best method for the small bacterial genome, ≈ 50% for the larger C. elegans genome, and only ≈ 40% for the human genome.k-mers that together hit all L-long DNA sequences. Such compact sets have many applications in sequence analysis, including space efficient data structures and large-scale sequence analysis. We tested the sets produced by our algorithm in an application that requires finding a small set of 10-mers hitting all 30-long words in the input genomes. Compared to minimizers, the current state of the art, our sets were almost 2.5-3.5 times smaller for the human genome. We could produce sets for the range of k = 5 to 10 and L = 20 to 200, and the results show that for L > 100 the size of the solution is quite close to the theoretical lower bound. We expect the sets produced by our approach to be useful and improving a variety of biological applications that require complex analysis of numerous sequences.We presented the DOCKS algorithm, which generates a compact set of k-mer counting applications, such as KMC 2 [k-mer hitting sets. Analyses that rely on k-mer counting, such as metagenomic binning as implemented in Kraken [k-mers can typically be used as a drop-in replacement, improving computational performance.We see the benefit of our compact UHSs in many data structures and algorithms that analyze high-throughput sequencing data. For example, we expect that binning-based as KMC 2 , can redn Kraken , will alL-long sequences. In the second phase we greedily remove k-mers that hit remaining L-long sequences. Overall efficiency is primarily due to the first phase, which runs in time O(k) times the size of the output. In the second phase dynamic programming is used, providing running time polynomial in the output size.The good performance of the algorithms can be attributed to their two phase approach. In the first phase we optimally and rapidly remove a minimum-size set that hits all infinite sequences, which also takes care of many k = 10, DOCKSany for k = 11, and the fastest variant, DOCKSanyX for k = 13 (with X = 10000) with L = 200, within a day. Note that all heuristics are bound to hit a limit since their runtime depends exponentially on k. This is an inherent property of the problem and its output size. Still, we manage to increase k by one or two using each heuristic. In partial remedy, we also proposed a construction that can push that limit further at the expense of solution size.We developed two additional variants of DOCKS that reduce the runtime and memory usage at the price of increasing the size of the set created. DOCKS can provide a solution for k, L)-hitting set for a given set of sequences is NP-hard. Moreover, even after removing an optimal decycling set, one needs to solve the problem of finding a minimum vertex set that hits all L-long sequences in a directed acyclic graph, which is NP-hard. Hence, DOCKS usually produces sub-optimal solutions. For example, for k = 4 and L = 10 the optimal solution obtained by solving an ILP formulation had size 89, compared to 91 produced by DOCKS. In fact, our tests show that if further reduction to the hitting set size is needed, starting from the DOCKS solution and improving it using ILP is a good strategy, at least for small values of k.Our approaches are heuristic in nature. This is not surprising, since as we show, the problem of finding a minimum -hitting set similar to the characterization of decycling sets by Mykkeltveit [k and L that can check if a k-mer belongs to a particular universal -hitting set. The fact that MINIMUM -HITTING SET on a given set of input sequences is NP-hard still leaves the universal case open. A related question is whether one can find an algorithm that generates an optimal  -hitting set while requiring work polynomial in the output set size. This is particularly interesting for the universal case, where the input is only the values k and L and the output size is > |Σ|k/k. Second, is the problem of minimum ℓ-path cover in a DAG G polynomial when G is a subgraph of a de Bruijn graph? We know it is hard for a general DAG, but the specific structure of de Bruijn graphs may make the problem easier. Third, the bottleneck to DOCKS running time is the second phase, which currently re-calculates the vertex hitting numbers on each iteration. Can one find a dynamic algorithm that updates these numbers more efficiently after the removal of one vertex? Fourth, is there a tight upper bound on the number p of vertices that will be removed by the greedy heuristic? Fifth, can we give an upper bound or a tighter lower bound on the size of Uk, L?Our study raises several open problems. First, is there a characterization for a minimum universal  pair, but the sets for all longer L values are computed when computing the  set with DOCKSany or DOCKSanyX and the runtime can be amortized across all of these calculations.The table contains solution set size, time in seconds and memory in KB for DOCKS, DOCKSany, DOCKSanyX and the greedy approach algorithms. Note that the reported times are for individual runs of each ((XLSX)Click here for additional data file.S1 Text(PDF)Click here for additional data file.


Scientific Reports 10.1038/s41598-017-18687-y, published online 18 January 2018Correction to: The original version of this Article contained an error in the title, where the phrase “bet hedging” was incorrectly given as “bet -hedging”. This has now been corrected in the PDF and HTML versions of the Article.


In this study, we present a novel model-based imaging method to enable high-resolution real-time PC-CMR with sparse sampling.Conventional phase-contrast cardiovascular magnetic resonance (PC-CMR) employs cine-based acquisitions to assess blood flow condition, in which electro-cardiogram (ECG) gating and respiration control are generally required. This often results in lower acquisition efficiency, and limited utility in the presence of cardiovascular pathology . Real-time PC-CMR, without ECG gating and respiration control, is a promising alternative that could overcome limitations of the conventional approach. But real-time PC-CMR involves image reconstruction from highly undersampled . Additionally, we performed a feasibility study of the proposed method on patients with cardiac arrhythmia.The proposed method achieves a spatial resolution of 1.8 mm and a temporal resolution of 18 ms for 2D real-time PC-CMR with one directional flow encoding. For the flow phantom experiments, both regular and irregular flow patterns were accurately captured. For the in vivo experiments with healthy subjects, flow dynamics obtained from the proposed method correlated well with those from the cine-based acquisitions. For the experiments with the arrhythmic patients, the proposed method demonstrated excellent capability of resolving the beat-by-beat flow variations, which cannot be obtained from the conventional cine-based method.The proposed method enables high-resolution real-time PC-CMR at 2D without ECG gating and respiration control. It accurately resolves beat-by-beat flow variations, which holds great promise for studying patients with irregular heartbeats.The online version of this article (doi:10.1186/s12968-017-0330-1) contains supplementary material, which is available to authorized users. Over the past few decades, phase-contrast cardiovascular magnetic resonance (PC-CMR) has been developed into a powerful tool for quantification and visualization of blood flow dynamics in the heart and large vessels –5. It haReal-time PC-CMR , 16 withIn this work, we present a new model-based method for real-time PC-CMR with sparse sampling. It is based on the integration of a novel low-rank model with parallel imaging. With temporal subspace modeling, the proposed method yields a convex optimization problem, thereby enabling efficient computation. The proposed method achieves real-time PC-CMR without ECG gating and respiration control, and well resolves the beat-by-beat flow variations that cannot be obtained from the conventional cine method. Compared with state-of-the-art real-time PC-CMR techniques, it provides higher temporal resolution. The effectiveness of the proposed method has been systematically evaluated in 2D real-time PC-CMR using both phantom experiments and in vivo experiments. A preliminary account of this work was presented in , 37.Ignoring flow during readout time, the imaging equation for real-time PC-CMR can be modeled as follows:ρv denotes the dynamic image associated with either the flow-compensated  or flow-encoded image sequence , Si(r) the sensitivity map for the i th receiver coil , and dv,i and ηv,i respectively the -space measured data and measurement noise. Here, the goal is to reconstruct ρv from the undersampled data {dv,i}, and then calculate the velocity maps as Δϕ = ∠ ρv − ∠ ρ1 denotes the phase difference between the flow-encoded and flow-compensated image sequences, and VENC the pre-specified encoding velocity. Since in real-time PC-CMR, there is no data sharing with ECG gating, -space data is often highly undersampled. Direct inversion of {dv,i} can incur significant aliasing artifacts and lead to inaccurate velocity measurements.where Cv admits a low-rank approximation due to strong spatiotemporal correlation of time-series images. Moreover, due to the nature of flow encoding, there is also strong spatial and temporal correlation among different flow image sequences. To exploit such correlation, the following joint Casorati matrix is introduced:Here we introduce a low-rank model-based reconstruction method with parallel imaging to address the problem. For convenience, we consider a discrete image model, in which each flow image sequence can be represented as a spatiotemporal Casorati matrix , i.e.,2\ications –41, eachC) ≤ L. There are a number of ways of imposing low-rank constraints [C = UV, where U ∈ ℂN × L and V ∈ ℂL × M. In this low-rank representation, the columns of U and rows of V respectively span the spatial subspace and temporal subspace of C.on which we enforce the low-rank structure, i.e., rank..di denoV by acquiring training data with a specialized data acquisition scheme. Specifically, as shown in Fig. , t)-space region with a random sampling scheme. With this sampling scheme, the two sets of data provide the complementary information for the low-rank model: the training data have high temporal resolution, while the imaging data have high spatial resolution. From the training data, we estimate the temporal subspace using the principal component analysis [U. To match the timing between the two sets of data, a proper temporal interpolation is performed, which interpolates the training data into those at the same time instants as the imaging data. Note that with such a scheme, the temporal resolution for the proposed method is 2 × Nv × TR. Moreover, note that the coil sensitivities Si can be estimated from temporal averaged -space data from the flow-compensated image sequence.The image reconstruction problem can be further simplified. Extending the early work in cardiac imaging , 41, 46,analysis , 47. WitU by solving the following convex optimization problem:Û, the joint Casorati matrix can be reconstructed as With entclass1pt{minimaWe performed both phantom and in vivo studies to evaluate the performance of the proposed method for 2D real-time PC-CMR. The experiments were conducted on a 3.0 T whole body MR scanner , equipped with a 32-channel cardiovascular coil. A gradient-echo (GRE) based pulse sequence was adapted to implement the proposed real-time acquisition scheme as shown in Fig. × 120 mm, matrix size = 182 × 100, spatial resolution = 1.20 mm × 1.20 mm, slice thickness = 5 mm, repetition time (TR) = 5.0 ms, echo time (TE) = 3.0 ms, flip angle = 10°, and VENC = 100 cm/s. Notice that the temporal resolution for the real-time acquisition is 4 × TR = 20 ms, while, for the cine acquisition, the temporal resolution is 56 ms (with 36 cardiac phases). The total acquisition time was around 42 s for both experiments.First, flow phantom experiments were performed to evaluate the capability of the proposed method in resolving various flow dynamics. Specifically, a 15-mm-diameter plastic tube simulating large vessel in the aorta was filled with blood-mimicking fluid , and plu× 1.80 mm, slice thickness = 5 mm, TR/TE = 4.5/2.8 ms, flip angle = 10°, and VENC = 200 cm/s. For the real-time flow imaging, the temporal resolution is 4 × TR = 18 ms, whereas for the cine imaging, the temporal resolution is around 36 ms (with 28 cardiac phases). The total acquisition time was around 94 s for both experiments.Second, in vivo experiments were performed to evaluate the proposed method. Ten healthy volunteers , who had no symptoms of cardiovascular diseases, were recruited. In addition, we performed a feasibility study of applying the proposed method for arrhythmia detection, and recruited two patients . This study was approved by the Institutional Review Board at Tsinghua University, and all the subjects gave written informed consent. Both the cine and real-time flow experiments were performed on the planes perpendicular to the ascending aorta (AAo) and descending aorta (DAo) during free breathing, and with one directional velocity encoding along the FH direction. For the cine acquisition, the retrospective ECG gating was set according to an estimate of each subject’s heartbeat period, and three averages were performed to mitigate respiratory motion artifacts. For both the cine and real-time imaging experiments, the following imaging parameters were used: FOV = 240 mm × 225 mm, matrix size = 132 × 124, spatial resolution = 1.80 mm V from the training data. We then estimated the coil sensitivity maps Si from the temporally averaged -space measurements. We further determined the spatial subspace U by solving Eq.  and 0.90 (P = 0.0002), respectively. This further confirms strong correlation between the two methods.We also performed a statistical analysis of the results from the two methods for all ten healthy subjects. Figure Figure Figure L = 20. As can be seen, the proposed method reconstructs the spatial images and velocity maps with improved quality over the alternative formulation. This illustrates the benefits of imposing the low-rank constraint on the joint Casorati matrix.Figure In this work, we introduced a new real-time flow imaging method and systematically demonstrated its effectiveness with both flow phantom experiments and in vivo experiments. Here, it is worth reiterating the key characteristics of the proposed method. First, the proposed method can be used as a viable alternative to the conventional cine flow imaging method in that it provides comparable (if not superior) image quality and flow information for healthy subjects. Second, the proposed method is able to resolve beat-by-beat physiological and/or pathological flow variations, which cannot be obtained from the conventional cine method based on ECG gating and respiration control. Such information is often clinically important .L). Generally, the selection of L needs to balance the model representational power, the number of measurements , and signal-to-noise ratio [L to trade off the above factors, and it consistently yielded good reconstruction performance, although it is worthwhile to investigate other principled model selection methods  in  in L). GThe proposed formulation results in a convex optimization problem, which enables efficient computation. For example, the runtime for reconstructing an in vivo dataset  takes around 10 min on a workstation with 64 GB RAM and 3.47 GHz CPU. The computational efficiency may be further improved by an implementation on graphical processing units. Such an investigation is beyond the scope of this paper, but is worthwhile to explore for future research.In addition to rank constraint, sparsity constraint can also be incorporated to accelerate PC-CMR. It has been demonstrated in , 43, 55 The flow-compensated and flow-encoded images share similar magnitude but different phase differences. We can extend the proposed method to exploit such information and impose a stronger constraint in the model-based reconstruction. However, the resulting formulation can involve a joint reconstruction of magnitude and phase images, which generally leads to a non-convex optimization problem. To solve such a problem, specialized algorithms and proper initialization are often needed. In contrast, the proposed method here employs a low-rank model to exploit the spatiotemporal correlation between flow images, which leads to a simple convex problem formulation and efficient computation. Given that the two models may have different trade-offs, comprehensively evaluating their advantages and drawbacks is a very interesting open problem to be explored in future work.In this work, we demonstrate the performance of the proposed method for 2D real-time flow imaging, in which through-plane flow was imaged. Considering the complex flow patterns and blood vessel geometry, it is highly desirable to perform 3D real-time flow imaging. However, 3D real-time flow imaging generally involves a more challenging trade-off between spatial resolution, temporal resolution, and imaging time, and a significantly more challenging computational problem. We are investigating an extension of the proposed method to 3D real-time flow imaging, and the results will be reported in future work.This paper is focused on the development of a novel real-time flow imaging technique, which should serve as a foundation for our subsequent clinical studies. Given that the proposed method well resolves beat-by-beat flow variations, it can provide more information on hemodynamics for patients with significant irregular heartbeats. In the future work, we plan to conduct systematic study of the proposed method for various potential clinical applications .It is also worthwhile to remark on the potential limitations of the proposed method. First, note that the aforementioned spatial and temporal resolution both refer to nominal resolution. For a linear shift-invariant reconstruction method , the resolution can be characterized through the point spread function. However, for a nonlinear reconstruction method -space data for real-time PC-CMR. The effectiveness and utilities of the proposed method have been demonstrated for 2D real-time PC-CMR with both phantom experiments and in vivo experiments. We expect that the proposed method will enhance the practical utility of real-time PC-CMR for various clinical applications.A new model-based method was introduced for high-resolution real-time PC-CMR without ECG gating and respiration control. It integrates the novel low-rank model with parallel imaging, which enables high-quality image reconstruction from highly undersampled (k


Scientific Reports 10.1038/s41598-017-17572-y, published online 12 December 2017Correction to: The original version of this Article contained an error in the title of the paper, where the word “Heart” was incorrectly given as “Heat”. This has now been corrected in the PDF and HTML versions of the Article, and in the accompanying Supplementary Information file.


The ToyBox-intervention is a theory- and evidence-based intervention delivered in kindergartens to improve four- to six-year-old children’s energy balance-related behaviours and prevent obesity. The current study aimed to (1) examine the effect of the ToyBox-intervention on increasing European four- to six-year-old children’ steps per day, and (2) examine if a higher process evaluation score from teachers and parents was related to a more favourable effect on steps per day.A sample of 2438 four- to six-year-old children  from 6 European countries  wore a motion sensor (pedometer or accelerometer) for a minimum of two weekdays and one weekend day both at baseline and follow-up to objectively measure their steps per day. Kindergarten teachers implemented the physical activity component of the ToyBox-intervention for 6 weeks in total, with a focus on (1) environmental changes in the classroom, (2) the child performing the actual behaviour and (3) classroom activities. Children’s parents received newsletters, tip cards and posters. To assess intervention effects, multilevel repeated measures analyses were conducted for the total sample and the six intervention countries separately. In addition, process evaluation questionnaires were used to calculate a total process evaluation score  for teachers and parents which was then linked with the physical activity outcomes.p > 0.05). In general, the intervention effects on steps per day were least favourable in four- to six-year-old children with a low teachers process evaluation score and most favourable in four- to six-year-old children with a high teachers process evaluation score. No differences in intervention effects were found for a low, medium or high parents’ process evaluation score.No significant intervention effects on four- to six-year-old children’ steps per weekday, steps per weekend day and steps per average day were found, both in the total sample and in the country-specific samples  contains supplementary material, which is available to authorized users. Sufficient levels of physical activity (PA) are associated with numerous positive mental and physical health outcomes, even at preschool age . In addiThe aim of the ToyBox-study was to develop, implement and evaluate a kindergarten-based, family-involved intervention to prevent overweight and obesity in four- to six-year-old preschool children from six European countries . One of the four behaviours on which the intervention focused, was PA  , 20. Thereach (level of participation in the intervention), fidelity , dose delivered (the amount of the intervention that was delivered by the implementers), dose received – exposure , dose received – satisfaction (the level of satisfaction of the implementer and the target group regarding the intervention) and context (within which the ToyBox-intervention was implemented) [A process evaluation should be guided by using a specific framework. The model used in the ToyBox-intervention is the model by Saunders et al. (2005) and is described in detail elsewhere , 26. Thiemented) .Thus, the aim of the current study was (1) to evaluate the effectiveness of the ToyBox-intervention on European preschoolers’ objectively measured steps per day in the total sample and in the country-specific samples, and (2) to examine whether a higher level of intervention implementation and satisfaction of the PA-component of the ToyBox-intervention was related to more favourable effects on preschoolers’ steps per day in the total sample.The kindergarten-based ToyBox-intervention with family involvement, targeting four- to six-year-old preschoolers, has a cluster randomised pre-test post-test design with intervention kindergartens and control kindergartens across six European countries: Belgium, Bulgaria, Germany, Greece, Poland, and Spain. Preschool children and their parents/caregivers were recruited at kindergartens, daycare centers or preschool settings, depending on the country regulations and legislation. In Germany, Bulgaria, Spain and Poland, children/families were recruited from kindergartens, in Greece from kindergartens and daycare centers, and in Belgium from preschool settings. In order to avoid confusion for the reader, all these settings  will be referred to as “kindergartens” in this paper. Additionally, to avoid confusion regarding the term “preschool children” we will use the term “four- to six-year-old children” throughout the manuscript.n = 16,798; age at baseline was 3.5–5.5 years old) received an information letter to take home in which the purpose of the study was explained to the parents/caregivers, and the child was invited to wear a pedometer or accelerometer for six consecutive days. Sample size calculations can be found elsewhere [Kindergartens were selected in the provinces of West- and Oost-Vlaanderen in Belgium, Varna in Bulgaria, Bavaria in Germany, Attica in Greece, Mazowiecki in Poland, and Zaragoza in Spain. Kindergartens were recruited from different SES backgrounds. Lists of all municipalities that exist within the selected provinces were collected and information on the SES variables was provided . Tertiles were created, based on the selected SES variables, and five municipalities were randomly selected from each tertile  in each country. Then, 1003 kindergartens within these randomly chosen municipalities across the six countries were randomly selected , and a personal visit was performed to inform the kindergarten staff about the ToyBox-study. In total, 309 kindergartens (30.8%) across the six countries agreed to participate in the study, and all four- to six-year-old children born in 2007 and 2008  to avoid contamination between kindergartens in the same municipality. This was done by by the project coordinator (Greece) with the use of a command in Excel, which means that the randomisation occurred automatically and electronically. Kindergartens allocated to the intervention condition received the intervention programme. Kindergartens allocated to the control condition continued with the normal kindergarten curriculum. The ToyBox-intervention was implemented within the school year 2012–2013.Before the start of the intervention, baseline measurements were performed on weekdays from May to June 2012. On those days, researchers visited intervention and control kindergartens and fitted those preschool children with an accelerometer (Belgium) or a pedometer (other countries) for whom written informed consent from their parents/caregivers had been obtained. One year later, from May to June 2013, follow-up measurements were performed and again, the same children with written informed consent from both intervention and control kindergartens received a pedometer or accelerometer to objectively measure their step counts.clinicaltrials.gov, ID: NCT02116296.The ToyBox-study was approved by the Ethical Committees in all European countries, in line with national regulations , and had a two-week ‘repetition period’ in weeks 19 and 20. Furthermore, some PA-components were also implemented throughout the whole school year, such as restructuring the classroom  in the classroom that stimulates moving such as balls, big sponge bricks,…) and installing movement landscapes  in the classroom. The ToyBox-intervention was implemented by the kindergarten teachers, who had two 1-h teacher training sessions with the researchers to explain the goals and the material of the ToyBox-study, and to answer kindergarten teachers’ questions, prior to the intervention. During the second training session, teachers were provided with the “ToyBox”, i.e. a box containing material for the class , and material for the home . In the classroom activity guide for PA, three different themes were included, namely (1) setting environmental changes in the classroom , (2) the preschool child performing the actual behaviour , and (3) classroom activities . Teachers were asked to allocate a minimum of 1 h per week to use the ToyBox-materials and to implement the ToyBox-intervention in the classroom, while the environmental changes were conducted and retained throughout the project. The 1 h per week recommendation was chosen based on the focus groups that were held before the development of the intervention . Kindergwww.toybox-study.eu).To involve the parents/caregivers, four- to six-year-old children received two newsletters, two tip-cards and one poster (with key messages on PA that could be coloured at kindergarten or at home) to take home for their parents/caregivers. The newsletters and tip-cards contained tips and strategies to increase four- to six-year-old children’ PA levels. The relevant materials can be found on the ToyBox-website   and ActiGraph  accelerometers with activated step count function (Belgium). Step counts from the pedometer and accelerometer are comparable and have been validated to measure PA in preschool children . The devFour- to six-year-old children wore a motion sensor (pedometer or accelerometer) for six full consecutive days, including two weekend days. Accelerometers were initialised to measure activity counts and step counts in 15-s epochs. Four- to six-year-old children’s parents/caregivers received an information letter and were instructed to let their child wear the motion sensor during all waking hours and to remove it only during water-based activities. After data collection, data from pedometers were downloaded using Omron Health Management Software version E1.012, and data from accelerometers were downloaded using ActiLife version 5.5.5-software. Both the first (fitting day) and sixth day (collection day) were omitted, because these days did not have a full day of data and were therefore incomplete. All step counts below 1000 and above 30,000 steps per day were deleted and treated as missing data, which is according to the rules of Rowe et al. (2014) . Four- tTo investigate the intervention process and satisfaction, teachers had to fill in two logbooks with specific questions regarding the process and satisfaction of the PA component, which were based on the model of Saunders et al. (2005) , 26. TheThe specific questions per key element that were used from the monthly logbooks are depicted in Table Also parents/caregivers had to fill in a questionnaire regarding the process of the PA component of the intervention. These questions were also based on the model of Saunders et al. (2005) , 26. TheThe specific questions per key element that were used from the questionnaire are depicted in Table Steps per day were separately calculated for weekdays, weekend days and average days. To take into account that some four- to six-year-old children had more weekend days than others, outcome variables on an average day were calculated using the following formula: *5) + *2)/7. Prior to all analyses, all outcome measures were first checked for normal distribution (skewness <0.70) and appeared to be normally distributed. Descriptive statistics were computed to describe the characteristics  of the sample, and were reported as frequencies (%) or means and standard deviations or percentages.Multilevel repeated measures analyses were performed using MLwiN 2.31  to assess the effectiveness of the intervention on steps per weekday, weekend day and average day. Multilevel modeling  was used to take clustering of two measurements (baseline and follow-up) of four- to six-year-old children in kindergarten classes in kindergartens in countries into account. For the country-specific analyses, four levels were used . All analyses were adjusted for age and sex. Two ß-values will be reported in the results: (1) the ß-value for ‘time’ is the estimate for the time effect, and can be interpreted as the magnitude of change in the outcome variable going from follow-up to baseline for the reference category , and (2) the ß-value for ‘time*condition’ is the estimate for the intervention effect for all outcome variables, which describes the difference between the mean change in the intervention group and the mean change in the control group.p < 0.05.To study if the effect was different regarding the intervention process evaluation score four- to six-year old children’s steps per weekday, steps per weekend day and steps per day, multilevel repeated measures analyses  were performed . All analyses were adjusted for children’s age and sex. For all analyses, statistical significance level was set at t = 8.07, p < 0.001) and average days . There was no significant difference for steps at weekend days. In addition, 37.4%, 28.9%, and 32.1% of four- to six-year-old children complied with the PA guidelines of 11,500 steps per day [In total, 2438 children  provided valid step count data at baseline and follow-up (drop-out of 71.3%). At baseline, four- to six-year-old children took 10,739 (±3258) steps per weekday, 9690 (±3850) steps per weekend day, and 10,440 (±2898) steps on an average day. At follow-up, they took 10,745 (±3501) steps per weekday, 9028 (±3924) steps per weekend day, and 10,254 (±3059) steps on an average day. Descriptive analyses showed that four- to six-year-old children from the control group significantly took less steps at baseline compared to four- to six-year-old children from the control group, both at weekdays  did not fill in the logbook for PA during the first focus, and 23.3% (n = 107) did not fill in the logbook for PA during the repetition period. The mean teachers’ process evaluation score for all intervention kindergartens was 13.49 (±2.56) on a total score of 26 . Kindergartens were divided into three groups, based on the teachers’ process evaluation score: (1) kindergartens with the lowest teachers’ process evaluation score , (2) kindergartens with a medium teachers’ process evaluation score , and (3) kindergartens with the highest teachers’ process evaluation score . Of all 312 teachers, 37.1% had a low, 29.7% had a medium and 33.2% had a high teachers’ process evaluation score. Teachers scored best on dose received  and fidelity (6.23/11) and had the lowest scores for dose delivered (1.55/6). Detailed information on the scores for each component of the teachers’ process evaluation score can be found in Table In total, a teachers’ process evaluation score could be calculated for 1605 children (out of 2438), as not all teachers filled in the monthly logbooks. From the 460 teachers who participated in the intervention, 8.1% (p = 0.02). Four- to six-year-old children with a low teachers’ process evaluation score significantly decreased their steps per weekday from baseline to follow-up , while children from the control group did not significantly change their steps per weekday from baseline to follow-up (p = 0.52). Furthermore, a significant interaction effect was found between children with a low teachers’ process evaluation score and a medium and high teachers’ process evaluation score, going from baseline to follow-up (β = 558.57 (SE = 250.95), p = 0.03 and β = 682.16 (SE = 264.62), p = 0.01). Four- to six-year-old children with a medium teachers’ process evaluation score and a high teachers’ process evaluation score did not significantly decrease their steps per weekday going from baseline to follow-up , while children with a low teachers’ process evaluation score significantly decreased their steps per weekday .Regarding steps per weekday, a significant interaction effect was found between children from the control group and children with a low teachers’ process evaluation score, going from baseline to follow-up (β = −553.95 (SE = 235.50); p = 0.003). Four- to six-year-old children from the control group significantly decreased their steps per weekend day from baseline to follow-up , while children with a high teachers’ process evaluation score did not (p = 0.99). Furthermore, a significant interaction effect was found between children with a low and children with a medium teachers’ process evaluation score (β = −669.56 (SE = 310.93), p = 0.03), and between children with a medium and children with a high teachers’ process evaluation score (β = 1095.61 (SE = 312.67), p < 0.001). Four- to six-year-old children with a medium teachers’ process evaluation score significantly decreased their steps per weekend day from baseline to follow-up , while children with a low teachers’ process evaluation score and children with a high teachers’ process evaluation score did not show a significant change in their steps between the two time points .Regarding steps per weekend day, a significant interaction effect was found between children from the control group and children with a high teachers’ process evaluation score, going from baseline to follow-up (β = 866.35 (SE = 293.65), p = 0.01). This means that children with a low teachers’ process evaluation score significantly decreased their steps per day from baseline to follow-up , while children with a high teachers’ process evaluation score did not change their steps per day from baseline to follow-up. No other interaction effects were found for steps per day. All interaction effects between time and teachers’ process evaluation score for steps per weekday, steps per weekend day and steps per day can be found in Fig. Regarding steps per day, a significant interaction effect was found between four- to six-year-old children with a low teachers’ process evaluation score and a high teachers’ process evaluation score, going from baseline to follow-up (β = 608.98 SE = 234.10); 4.10; p =four- to six-year-old children = 464), (2) four- to six-year-old children with a medium parents’/caregivers’ process evaluation score , and (3) four- to six-year-old children with a high parents’/caregivers’ process evaluation score . Of all parents, 31.8% had a low, 34.7% had a medium, and 33.5% had a high parents’/caregivers’ process evaluation score. Parents/caregivers scored best on dose received - exposure (4.86/10) and had the lowest scores for dose received - satisfaction (1.88/6). Detailed information on the scores for each component of the parents’/caregivers’ process evaluation score can be found in Table In total, a parents’/caregivers’ process evaluation score could be calculated for 1420 children (drop-out of 11.5%). The mean parents’/caregivers’ process evaluation score for all intervention kindergartens was 6.90 (±3.50) on a total score of 17 . Four- to six-year-old children were divided into three groups, based on the parents’/caregivers’ process evaluation score: (1) four- to six-year-old children with a low parents’/caregivers’ process evaluation score . This means that there is no difference between baseline and follow-up in steps per weekday, steps per weekend day or steps per day for four- to six-year-old children from the control group and four- to six-year-old children with a low, medium and high parents’/caregivers’ process evaluation score.For steps per weekday, steps per weekend day and steps per day, no significant interaction effects were found between time and parents’/caregivers’ process evaluation score . Second, we wanted to examine whether a higher teachers’ or parents’/caregivers’ process evaluation score was related to more beneficial effects on four- to six-year-old children’s steps per day.For the total sample, no intervention effects were found, which means that the intervention did not cause an effect on four- to six-year-old children’s PA in terms of steps per day. Also country-specific analyses revealed no effects on European four- to six-year-old children’s steps per day. The lack of effects on children’s PA is in line with other published studies targeting four- to six-year-old children’s PA levels , 17. OneThe results from the process evaluation showed a medium teachers’ process evaluation score with a mean score of 13.5 on a total of 26. The higher score on fidelity can be explained by the fact that almost all kindergarten teachers handed out the newsletters and tip-cards for the parents/caregivers, based on the set time plan of the ToyBox-intervention. In addition, the higher score for dose received (satisfaction) shows us that kindergarten teachers in general might have enjoyed the intervention components and that they might be convinced about the clarity, appropriateness and the feasibility of the intervention components. The low score for dose delivered can be explained by the fact that the classroom activities were not implemented as intended. Although kindergarten teachers read the Kangaroo stories during the first weeks of intervention implementation, most of them did not implement the proposed excursions. In addition, during the repetition period, less than 25% of kindergarten teachers repeated using the Kangaroo stories and more than 90% of all teachers did not execute the excursions. This shows that excursions might be hard to implement in kindergarten classes, and that the Kangaroo stories can be used for a short period during the intervention, but not for longer implementation periods.The results in this paper show that there are some differences in effect between the control group and between kindergartens with different process evaluation scores (low to high). For example for steps per weekday, four- to six-year-old children with a medium and high process evaluation score did not decrease their steps from baseline to follow-up, while four- to six-year-old children with a low process evaluation score did. This shows that having teachers that do not implement the intervention as intended  and were not satisfied with the intervention, could have unfavourable effects compared to teachers that implement the intervention in a better way and were satisfied with the intervention or even compared to teachers that did not implement the intervention at all . This shows that not implementing an intervention  is better than implementing an intervention with low fidelity to the study protocol. The influence of teachers implementing the intervention as intended  was still present on weekend days, as children from the control group decreased their steps per weekend day from baseline to follow-up, while children with a high teachers’ process evaluation score did not. This shows that teachers implementing the intervention with high fidelity to the study protocol can stop a possible decrease in steps per weekday or weekend day. These age-related declines in PA have been studied before  and coulA low parents’/caregivers’ process evaluation score was found with a mean score of 6.90 on a total score of 17. The results suggest that parents/caregivers read and were satisfied with the newsletters, tip-cards and posters for the PA component of the ToyBox-intervention, but that they were not satisfied with for example the usefulness, design and the amount of text of these tools. In addition, results showed that there was no difference in effect between four- to six-year-old children from the control group and four- to six-year-old children with a low, medium or high parents’/caregivers’ process evaluation score. This is comparable with the IDEFICS-intervention in which higher levels of parental exposure to the intervention was not related to more favourable effects in children’s BMI z-scores, and parental exposure and involvement in the intervention was much less than aimed for . In the Based on the information received from both kindergarten teachers and parents/caregivers, the ToyBox-material and/or components could be adapted and revised to meet the comments. For example by providing active toys and/or activity videos for the teachers. In addition, qualitative research could be helpful to investigate kindergarten teachers’ and parents’/caregivers’ perspectives on what worked and what did not work. The revised material and/or components could then be pilot tested to see whether teachers and parents/caregivers are more satisfied with the intervention components and implement the intervention as intended.Study strengths are the large sample of four- to six-year-old children that provided valid pedometer data, and the cluster randomised controlled trial with a pre-test post-test design. However, future studies could look into objectively measuring four- to six-year-olds’ PA by using hip and wrist accelerometers simultaneously as wrist-worn accelerometers have shown to measure different PA intensities more accurately . AnotherStudy limitations include the fact that pedometers cannot distinguish between the different PA intensities and are only able to measure in steps per day. Missing data  were not imputed, which may have induced bias. Furthermore, there was a large drop-out of four- to six-year-old children due to a lack of valid pedometer data at both baseline and follow-up. Also, there was no objective measurement of intervention implementation as the data for the process evaluation scores were self-reported in both teachers and parents/caregivers. It is possible that both teachers and parents/caregivers overestimated their efforts, which might have introduced bias. In addition, as not all teachers filled in all logbooks and not all parents filled in the process evaluation questionnaire, this also might have induced bias. Future studies could try to incorporate objective methods  to study process evaluation. Furthermore, teachers’ logbooks did not contain specific questions regarding the implementation of the environmental changes in the classroom, which means that valuable information was not recorded. This could however be taken into account by using observations. Another limitation that should be considered is that the differences by countries regarding the process evaluation score and sub-scores of each process evaluation component were not reported. However, this manuscript already gives a general image of the link between effect and process evaluation for all countries combined. Finally, it should be acknowledged that there is no standardised way to calculate process evaluation scores. Although it was based on a method used in previous studies , 49, 50,Overall, the ToyBox-intervention had no effect on European four- to six-year-old children’s steps per day. Future European interventions with the aim to increase four- to six-year-old children’s steps per day should incorporate country-specific adaptations to increase the effectiveness. The process evaluation scores showed that higher teachers’ process evaluation score did not cause a decrease in steps per day, while four- to six-year-old children with a lower score or from the control group did. Therefore, future interventions should search for strategies to motivate the kindergarten teachers to induce larger effects, for instance by using a participatory approach in which kindergarten teachers are involved from the beginning of the study or by providing more active toys and exercise videos. However, this should be combined with actively involving children’s parents.


TATES is a so-called combination test: it tests a multivariate hypothesis by combining the m p-values obtained in the m univariate tests. Specifically, TATES is based on selection of the minimal p-value among m appropriately weighted p-values, and as such tests the hypothesis that at least one of the m phenotypes is associated to the particular SNP. TATES was inspired by the GATES procedure, a gene based test of association  given two phenotypes.We gratefully use this opportunity to comment on this work.m equaled 20, the number of simulations Nsim equaled 2000, and α was set to 0.05. These simulation settings were deemed realistic in the context of questionnaire data , and tailored to this context, featuring various realistic models to account for the phenotypic covariance structure .In the original TATES paper, the authors showed in 20 scenarios (8 of which concerned the effect of missing data) that the Type I error rate of TATES is correct when the number of phenotypes m = 2 and m = 3 phenotypes in 10 correlational settings. In these simulations, 6 out of 10 (m = 2), and 21 out of 30 (m = 3) phenotypic correlations r > 0.70 . They then show that, given Nsim = 100,000, the Type I error rate of TATES deviated significantly from 0.05 ) in 8 of the 10 scenarios when m = 2, with a maximal empirical rate of 0.0553 (when r = 0.9343) instead of expected 0.05. When m = 3, the Type I error rate of TATES deviated significantly from 0.05 in all 10 presented scenarios, with a maximal empirical rate of 0.0540 . Before we dwell on the statistical and practical significance of deviations of this magnitude, we first wish to gain a more comprehensive view of TATES’s Type I error rate.Aliev et al. report simulations featuring m and the correlations r between the m phenotypes. Specifically, we simulated data for m = 2, 4, 8, and 16, and r = 0.1, 0.3, 0.5, 0.7, and 0.9, resulting in 20 simulation settings in total. Note that the resulting correlational structure is compound symmetric , which is consistent with a single  factor model. We simulated phenotypic and genotypic data for N = 2000 subjects. Like Aliev et al., we simulated a single diallelic variant , and ran Nsim = 100,000 simulations for each setting.To investigate the empirical Type I error rate of TATES more extensively, we ran additional simulations in which we varied both the number of phenotypes m weighted p-values.To broaden the present scope of our simulations and to put the TATES results into perspective, we analyzed the simulated data using TATES and three other combination tests that, like TATES, are based on selection of the minimal p-value among Bonf; Simes m univariate analyses to regress m phenotypes on a SNP, the m p-values are all Bonferroni-corrected , and then the smallest Bonferroni-corrected p-value is selected. The second combination test, which we refer to as minPNS, is similar to minPBonf, except that one does not correct for the observed number of phenotypes m, but for the effective number of phenotypes Meff. As suggested by Nyholt  and the highest p-value is weighted with the smallest weight . The Simes p-value then corresponds to the smallest weighted p-value.The third combination test is the original Simes test that TATES is a variation on Simes . In Simem and j are replaced with the effective number of p-values me and mej. Specifically, TATES weights each jth p-value pj by me/mej, and mej is calculated asj is the number of top j p-values, λi denotes the ith eigenvalue of the correlation matrix between the j p-values (which can be approximated from the correlations between the j phenotypes), and I(x) is an indicator function taking on value 0 if λi ≤ 1 and 1 if λi > 1. That is, the effective number of p-values mej among the j p-values is calculated as the observed number of p-values j minus the sum of the difference between the eigenvalues λi and 1 for those eigenvalues λi > 1. The value of me is equal to mej for the case that j = m, i.e., when the selection of top p-values covers all p-values. The TATES p-value then corresponds to the smallest weighted p-value. Following this procedure, the smallest original p-value is always weighted by the largest weight, while the largest original p-value is weighted by me/me = 1 as in that case mej = me. As the weight me/mej is always ≥ 1, the weighted p-values are, like in the Simes test, always ≥ the original, unweighted p-values.TATES, which is based on GATES .All four combination tests test the hypothesis that at least 1 of the 95 given α = 0.05. The standard error of the ML estimator of the p-value is p denotes the percentage of significant tests expected given the chosen α , and Nsim the total number of simulations. Given Nsim = 100,000 and α = 0.05, the CI95 thus equals  = . In Table 95 given Nsim = 100,000 are italicized, while values falling outside the CI95 given Nsim = 10,000 are italicized and bold.We ran the 20 simulation scenarios for all four methods, and then established the percentage of p-values per scenario smaller than 0.05. For all four methods, these observed Type I error rates are shown in Table mentclass2pt{minimm is small, yet slightly conservative when m is large. Simes and minPBonf are almost always conservative, and becomes more so with increasing correlations and increasing m. In contrast, minPNS is generally liberal, and especially so when m is small and correlations are high. If we sum the absolute deviations from 0.05 across all 20 scenarios, we see that overall, TATES remains closest to 0.05, while minPBonf shows the largest overall deviation, entirely due to its conservativeness. Indeed, minPBonf shows the largest undershoot, while minPNS shows the largest overshoot.The Type I error results in Table m- and r-dependent variation around 0.05, but these deviations are small, especially compared to the other considered combination tests. The deviations reported by Aliev et al. are associated with the special case of small m and (very) high r. However, regardless of the narrow scope of the simulations by Aliev et al., one may still ask whether the observed deviations are a reason to reject the TATES procedure.Overall, the Type I error rate of TATES does show Nsim = 2000 , Nsim = 10,000, and Nsim = 100,000 , the CI95’s of an unbiased p-value are 0.0404–0.0596, 0.0457–0.0543, and 0.0486–0.0514, respectively. The empirical Type I error rates of TATES, as displayed in Table Nsim. Aliev et al. emphasized statistical significance in assessing the Type I error results. However, in this situation, we believe that it is more important to consider the practical relevance of deviations of 0.05, 0.005, or 0.001. We are convinced that deviations of this magnitude, while statistically significant given large Nsim, cannot justify the rejection of TATES, or any other method, and believe therefore that the TATES Type I error rates give little reason for concern.The larger the number of replications in a simulation study, the more power one has to demonstrate that the empirical Type I error rate of a method deviates from the expected Type I error rate. For instance, with Bonf, minPNS, and TATES work with these p-values. However, these combination tests themselves are based on selection of the minimal p-value among m weighted p-values. So while the p-values of the m univariate tests, on which these combination tests are based, should indeed generally be uniformly distributed, the relevant question here is whether the p-values resulting from combination tests’ weighted-selection procedure should be uniformly distributed as well.Aliev et al. argued that the distribution of the TATES p-values should be uniform under the H0, and that the probability distribution function should at least not exceed 1 at the low end of the distribution, i.e., around 0. They consider this a condition for the Type I error rate of TATES to be correct. It is important to note that p-values may not be uniformly distributed under the H0 in special cases  in each of the 20 aforementioned simulation settings .Figure m p-values that they are based on are. Specifically, when all p-values are weighted with the same weight , selection of the minimal weighted p-value results (as could be expected) in a p-value distribution with a right, positive skew. When smaller p-values are weighted more heavily, the deviation of uniformity increases with increasing m and increasing correlations, with the bulk of p-values at the high end of the distribution. Indeed, as both Simes and TATES weight the highest original p-value with the smallest weight , the Simes p-value and the TATES p-value very often equal the largest p-value before weighting.Clearly, p-value distributions of methods that are based on selection of the minimal weighted p-value are not uniformly distributed under H0, even if the or deflation, depending on the number of phenotypes m and the strength of their intercorrelations r. However, we consider the observed deviations of 0.05, whether or not statistically significant, to be too small to be considered of practical concern. Note that in this commentary, we addressed the Type I error rate given an expected rate of α = 0.05, like Aliev et al. did. We also considered α = 0.01 and α = 0.001  and found that Type I error rates of TATES were close to expectation . While again some values deviated statistically significantly from expectation , the deviations were small and, in our view, of little practical significance.Aliev et al. set out to show that the Type I error rate of TATES procedure is incorrect by presenting results of a simulation study and a mathematical proof concerning the non-uniformity of TATES’s p-value distribution. With respect to the former, we believe that the simulation results of Aliev et al. as well as our own showed that TATES’s Type I error rate indeed shows slight in- With respect to mathematical proof concerning the uniformity of the p-value distribution, we believe that the assumption that the distribution of p-values of a method that is based on p-value selection should be uniform, is based on misconception. The p-values that the combination tests are based on should be uniformly distributed, but the p-values of subsequent weighted-selection-based combination tests are not expected to be uniformly distributed.All in all, if one wishes to apply a combination test to tackle a multivariate problem, we believe that TATES represents a viable option.Below is the link to the electronic supplementary material.Supplementary material 1 (DOCX 19 KB)


Scientific Reports 10.1038/s41598-017-03116-x, published online 07 June 2017Correction to: The supplementary file “Bayesian phylogeographic recostruction.kmz” was omitted from the original version of this Article. This has been corrected in the HTML version of the Article; the PDF version was correct at time of publication.


Zhang  Zhang et al. proved that the three-term PRP method has global convergence under Armijo line search technique for general functions but this fails for the Wolfe line search. The reason may be the trust region feature of the search direction that cannot be satisfied for this method. In order to overcome this drawback, we will propose a modified three-term PRP formula that will have not only the sufficient descent property but also the trust region feature.We consider the optimization models defined byentclass1pt{minimae, e.g., ). The CGlas see –46) and  and 1.1\e [e.g., –50), whee [e.g., , 51 defin et al.  proved tn et al. , Gilbertn et al. , Toouatin et al. , and Hu n et al.  hinted tIn the next section, a modified three-term PRP formula is given and the new algorithm is stated. The sufficient descent property, the trust region feature, and the global convergence of the new method are established in Section Motivated by the above observation, the modified three-term PRP formula is between  and Step 0:Initial given parameters: Step 1:Step 2:Get stepsize Step 3:Let Step 4:Calculate the search direction }dk+1 by .Step 5:Set It has been proved that, even for the function ice  Equation  is the sk.(2) Equations  and 2.22.2 implymentclass2pt{minimTo establish the global convergence of Algorithm (i)The defined level set (ii)f has a lower bound and it is differentiable. The gradient g is Lipschitz continuousThe function Suppose that Assumptionholds and NTT-PRP-CG-A generates the sequenceThen there exists a constantsuch thatUsing  and 2.32.3 generows from . By , , 3.1), aet al. [Himmelblau rule: Set This section will report the numerical experiment of the NTT-PRP-CG-A and the algorithm of Zhang et al.  (called }dk+1 by . Since t}dk+1 by , we onlynt}αk in  and (2.3Table No. the test problems number. Dimension: the variables number.Ni: the iteration number. Nfg: the function and the gradient value number. CPU time: the CPU time of operating system in seconds.A new tool was given by Dolan and Moré  to analyIn this paper, based on the PRP formula for unconstrained optimization, a modified three-term PRP CG algorithm was presented. The proposed method possesses sufficient descent property also holds without any line search technique, and we have automatically the trust region property of the search direction. Under the Wolfe line search, the global convergence was proven. Numerical results showed that the new algorithm is more effective compared with the normal method.


Feature selection is an important topic in bioinformatics. Defining informative features from complex high dimensional biological data is critical in disease study, drug development, etc. Support vector machine-recursive feature elimination (SVM-RFE) is an efficient feature selection technique that has shown its power in many applications. It ranks the features according to the recursive feature deletion sequence based on SVM. In this study, we propose a method, SVM-RFE-OA, which combines the classification accuracy rate and the average overlapping ratio of the samples to determine the number of features to be selected from the feature rank of SVM-RFE. Meanwhile, to measure the feature weights more accurately, we propose a modified SVM-RFE-OA (M-SVM-RFE-OA) algorithm that temporally screens out the samples lying in a heavy overlapping area in each iteration. The experiments on the eight public biological datasets show that the discriminative ability of the feature subset could be measured more accurately by combining the classification accuracy rate with the average overlapping degree of the samples compared with using the classification accuracy rate alone, and shielding the samples in the overlapping area made the calculation of the feature weights more stable and accurate. The methods proposed in this study can also be used with other RFE techniques to define potential biomarkers from big biological data. Feature selection is one of the main data analysis techniques in data mining, which has shown its power in many applications, such as insulator detection , medicinAlong with the quick development of the high throughput techniques, genomics, metabolomics, and proteomics have been widely applied in disease study, drug research, etc. One characteristic of omics data is summarized in the expression ”high dimensions, small samples”, since omics data usually have a large number of features but few samples. As genomics data, metabolomics data and proteomics data usually contain many features, it has been critical to accurately measure the feature importance and select the most discriminative feature subset. Puthiyedth et al.  presenteSupport vector machine (SVM)  is a popBasically, SVM-RFE ranks the features according to the feature deletion order during the iterations. The top ranked features which are removed in the last iteration of SVM-RFE are the most important, while the bottom ranked ones are the least informative and removed in the first iteration. For a specific application, it is not enough to obtain a feature importance ranking; it needs to determine how many top ranked features (such as genes and metabolites) should be selected. Thus, based on the selected features, we can study the disease phenotype and disease mechanism. In some studies, the top ranked features were selected according to a predetermined number ,24. In oIt is not practical to specify the number of features to be selected in advance in some applications. However, it is well known that the feature subset selected should have a powerful discriminative ability. If a feature subset has a powerful discriminative ability, then the classifier based on it usually has a high prediction accuracy rate, and the different sample groups on the selected subspace should show different distributions with little overlapping areas. Hence, this study proposes a method, SVM-RFE-OA, which determines the number of features to be selected from the feature rank of SVM-RFE by combining the classification accuracy rate and the average overlapping ratio of the samples together. In addition, to weigh the features more accurately, this study also proposes a modified SVM-RFE-OA (M-SVM-RFE-OA) algorithm, which temporally screens out the samples lying in a heavily overlapping area in each iteration. The experiments on the eight public biological datasets show the validation of the two techniques proposed.X = {n samples, C be the class label set, Label ≠ Label ≠ Label represents the degree that sample x mixes with heterogeneous samples, and the average Nr(x) of all samples in the dataset reflects the mixing degree of the different class samples on the current subspace. If different classes show different distributions on the current feature subspace, then there is a clear separation among different classes, and the average Nr(x) of all samples is small. If different classes show almost the same distribution, they mix together on the subspace, and the average Nr(x) of all samples is large. Hence, the average Nr(x) can express how much discriminative information the current feature subset contains.Therefore, F contains all the input features. In each loop, an SVM learning model is built based on the current feature subset F, the weight (|w|) of each feature in F is calculated according to the support vectors on the hyper-plane of the SVM classifier. The features are then ranked based on |w|, and the bottom ranked features are removed from F. This procedure is repeated until F is empty. The feature removing sequence represents the feature importance rank [F, the more important the features are. The top ranked features are those that are removed from F in the last iteration of SVM-RFE.SVM-RFE ,12 is a nce rank ,12. The F. When the procedure terminates, the feature subset corresponding to the maximal accuracy rate is kept as the final selected feature subset [Thus, we can obtain a feature rank via SVM-RFE. However, for a certain data analysis, how many top ranked features should be selected from the feature rank of SVM-RFE is still to be considered. In some cases, the number of features to be selected is decided according to prior knowledge or is simply decided subjectively . In othee subset ,16.d-fold cross validation on the training dataset can be applied to determine which feature subset is selected during the backward feature deletion, i.e., the number of the selected features is determined by the classification accuracy rate on the training dataset. However, classification accuracy reflects the discriminative ability of the feature subset based on the classifier, and the distribution of the samples can also reflect the discriminative ability of the feature subset. If different class samples mix together on the current subspace, the overlapping degree of the samples is large, and the subspace has little discriminative information. Both the classification accuracy and the overlapping degree of the samples can tell us how much discriminative information the feature subset has. They evaluate the feature subset from two different aspects, respectively. The discriminative ability of the feature subset could be evaluated more comprehensively by combining these two terms. Hence, we propose SVM-RFE-OA (see Algorithm 1 SVM-RFE-OA), which measures the feature subset during the iterations of SVM-RFE by integrating the average overlapping degree of samples and the classification accuracy rate, and selects a feature subset that has a large accuracy rate and a small overlapping degree. In each iteration, SVM-RFE-OA calculates the average accuracy rate (T_c_acc) of d-fold cross validation and the average Nr(x) (T_c_oa) of all the samples in the training data, and the feature subset having the largest “T_c_acc − T_c_oa” is kept as the final selected feature subset.Algorithm 1 SVM-RFE-OAX, t.Input: training dataset FS.Output: selected feature subset Beginc_acc = 0;  c_oa = ∞;  F = {all input features};  F| > 0) Do  While  of the samples in X based on F;   F by |w| in descending order;   Rank the features in T_c_acc − T_c_oa > c_acc − c_oa Then   If  c_acc = T_c_acc;     c_oa = T_c_oa;     FS = F;         Endif;F = F− {t × |F| bottom ranked features in F};        Endwhile;FS;  Return End.In biological data analysis, defining the most informative features (such as genes and metabolites) from the large complex data is of great importance to disease diagnosis and drug study. SVM-RFE is very efficient in analysis of large complex data. However, it is quite difficult to predetermine how many top ranked features should be selected from the feature rank of SVM-RFE. The classification accuracy rate of t (0 < t < 100%) is the filter factor. In each iteration of SVM-RFE, t × |F| bottom ranked features are removed from the current feature subset F.Nr(x) of each sample in the training data is calculated based on the current subspace F; (2) the samples with Nr(x) > 0 are temporarily set aside and are not used in SVM training in this iteration. At most, one-third of the samples in each class in the training data are screened out to make sure that there are enough samples kept for the training. Since the samples in the heavy overlapping area are shielded in the training procedure, there is little chance that overfitting occurs, and the bias becomes small.Algorithm 2M-SVM-RFE-OAX, t.Input: training dataset FS.Output: selected feature subset Beginc_acc = 0;   c_oa = ∞;  F = {all input features};  F| > 0) Do  While (|Nr(x) for each x ∈ X based on F;   Calculate t = XX;   c ∈ C Do   For each cX = {x | x ∈ X, Label(x) = c and Nr(x) > 0};    θ = |{x | x ∈ X, Label(x) = c}|;    cX| > θ/3 Then    If |cX based on Nr(x) in descending order;     Rank the samples in tX = tX − {θ/3 top ranked samples in cX};         Else t = Xt − XcX;         End if;   End for;tX and F;     Construct an SVM based on T_c_acc = accuracy rate of SVM;     T_c_oa = average Nr(x) of the samples in tX based on F;     T_c_acc − T_c_oa > c_acc − c_oa Then     If   c_acc = T_c_acc;     c_oa = T_c_oa;     FS = F;         End if;F by |w| in descending order;    Rank the features in F = F − {t × |F| bottom ranked features in F};      End while;FS;  Return EndIn the calculation of feature weights, only the samples on the hyper-plane of the SVM learning model are considered ,12. The http://ligarto.org/rdiaz/Papers/rfVS/randomForestVarSel.html, and the last two datasets are from www.gems-system.org. Breast2 [To show the performance of the two techniques proposed, SVM-RFE-OA and M-SVM-RFE-OA were compared with SVM-RFE where the selected feature subset was determined by the classification accuracy rate. Eight public biological datasets were used in the comparison of the three algorithms, where Breast2, Colon, Lymphoma, Prostate, Brain_data, Srbct are from  Breast2 ,28 conta Breast2 ,29 datas Breast2  dataset  Breast2 ,31 datas Breast2 ,32 conta Breast2 ,33 conta Breast2  dataset  Breast2 ,34 datashttp://www.csie.ntu. edu.tw/~cjlin/libsvm/. Linear kernel was adopted in the SVM, and t was set to 5%. In SVM-RFE-OA and M-SVM-RFE-OA, k was set to 9. Five-fold cross validation was run 50 times for each method. The average classification accuracy rates and the average standard deviations are given in SVM-RFE, SVM-RFE-OA, and M-SVM-RFE-OA were implemented in C++. SVM was obtained from First, we compared SVM-RFE-OA with SVM-RFE to examine the performance of SVM-RFE-OA. Secondly, we compared M-SVM-RFE-OA with SVM-RFE-OA, thereby examining the performance of temporally screening out the poor samples lying in an overlapping area. Both M-SVM-RFE-OA and SVM-RFE-OA combine the classification accuracy rate and the average overlapping degree to calculate the discriminative ability of the feature subset and determine the number of top ranked features to be selected. To measure the feature importance more accurately, M-SVM-RFE-OA temporarily shields the samples in the overlapping area in each iteration. The comparison between M-SVM-RFE-OA and SVM-RFE-OA shows that temporarily screening out the samples mixed with heterogeneous samples in each iteration benefits the calculation of feature weights. The comparisons between SVM-RFE and SVM-RFE-OA and between SVM-RFE-OA and M-SVM-RFE-OA validate the two techniques proposed in this study. Finally, it can be seen that M-SVM-RFE-OA outperforms SVM-RFE for all eight datasets in terms of accuracy rate and outperforms SVM-RFE for all the four binary datasets in terms of sensitivity and specificity. Especially for Brain_data, the accuracy rate of M-SVM-RFE-OA is higher than that of SVM-RFE by 10.2%.Meanwhile, SVM-RFE-OA and M-SVM-RFE-OA are more stable than SVM-RFE. The standard deviations of M-SVM-RFE-OA on accuracy rate, sensitivity, and specificity are lower than those of SVM-RFE in most cases. Hence, from two different aspects, the classification accuracy rate and the average overlapping degree of samples, which reflects the sample distribution on the feature subspace (top ranked feature subset), we can obtain a more comprehensive measurement of the feature subset. Further, temporally shielding the samples with high overlapping ratios in each iteration could make the computation of feature importance more accurate.In systems biology, it is very significant to select the most meaningful features from large complex genomics, metabolomics, and proteomics data, which could help in classifying different disease samples, studying disease mechanisms, and developing new drugs. This paper proposes two techniques of selecting discriminative feature subsets based on SVM-RFE. One is measuring the feature subset by combining the classification accuracy rate with the average overlapping degree of samples, and the other is temporally screening out the samples in a heavily overlapping area in each loop of the SVM-RFE. Experiments on eight public biological datasets show the validation of these techniques and prove that filtering out the samples that lie in the heavily overlapping area could make the measurement of feature weights more accurate.


There is an error in the 7th author’s name. The correct name should be Antonio Condino-Neto. As a result, the name is indexed incorrectly. The name should be indexed as Condino-Neto A.


In this paper, the authors study the boundedness of multilinear Calderón-Zygmund singular integral operators and their commutators in generalized Morrey spaces. T be a multilinear operator initially defined on the m-fold product of Schwartz spaces and taking values into the space of tempered distributions, i.e.,K belongs to the class m-if  The operator T is said to be an m-linear Calderón-Zygmund operator if there exists a function T extends to a bounded multilinear operator from Let mentclass2pt{minimm-linear Calderón-Zygmund operator satisfiesIt was shown in  that if m-linear Calderón-Zygmund operators was developed by Lerner et al. [B is the ball in The theory of multiple weight associated with r et al. . Let \dom-linear Calderón-Zygmund operator T and a b⃗ is defined byT in the jth entry of T, that is,Let entclass1pt{minimad Torres  proved tentclass1pt{minimaω be a weight. The space f satisfying Feuto  introducduced in . If \docSimilarly, the weak space duced in .Feuto proved in  that CalIn this paper, we aim to study the boundedness of multilinear singular integral operators on the product of generalized Morrey spaces. Inspired by the above mentioned works, we state our main results as follows.LetTbe an m-linear Calderón-Zygmund operator, andIfandthenIfand at least one ofthenLetbe a multilinear commutator, withandIfthenWhen ω belongs to p, i.e., B. If E of a ball B. Since the p, we use the following notation C is a positive constant independent of the main parameters. For λB for the ball with same center as B and radius λ times radius of B.We first recall the definition of Now we give the definition of p. When Let Obviously, if LetThenif and only ifwhere the conditionis understood asin the caseAssume thatsatisfiescondition. Then there exists a finite constantsuch thatBMO. For B,k, we obtainIn order to prove the results for commutators, we need the following properties of E. For III. Since II, we just consider this case: (1) Let sides of , we get(2) For }λ>0, by  and , , \documesides of , we have