Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent’s local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots. The transportation problem The transportation of a hose by a team of robots is a paradigmatic instance of the tasks that can be performed with Multi-Component Robotic Systems (MCRS)  is a probabilistic transition function P giving the probability of observing state s′ after executing action a in state s, and R:S → ℝ is the expected immediate reward after reaching state s. The set of terminal states 𝓣 is defined as the set of states having null outgoing transition probabilities, i.e. 𝓣 = {sS∣∀s′ ≠ s, a;P = 0}. The action selection policy is usually modeled as the probability distribution π:S × A →  of taking action a in state s.In RL algorithms, the interaction between an agent and its environment is modeled as a Markov Decision Process (MDPs), defined as a tuple < valueVπ(s) of state s is the expected accumulated reward obtained from that state following policy π. It can be expressed asEπ denotes the expected return from time step t following policy π thereafter, ri represents the reward received at time i, and γ ∈  is a discount-rate parameter decreasing the weight of the reward at each time-step. Bellman equation forVπ  is the probabilistic state transition function P that defines the probability of observing state s′ after all agents execute the joint action a ∈ A in state s, and R:S × A × S → ℝN are the expected rewards received by agents after transition .When the system is composed of several interacting autonomous agents, the Multi-Agent Reinforcement Learning (MARL) problem consists of the search for the joint optimal policies maximizing the reward for each and all agents . In the an = T ∈ A, ai, k ∈ Ai. As a consequence, the rewards also depend on the collective action. The local policy of each agent πi:S × Ai →  gives the probability of the ith agent executing action a in state s. The local policies are composed into a global joint policyπ = {π1, …, πN}.In the multi-agent case, state transitions are the result of the concurrent joint action cooperative when the local rewards are identical for all agents R1 = R2 = … = RN,The SG is fully ai ∈ Ai and i = 1, …, N. When there are more than one optimal action choice, the optimal policy is achieved by a consensus protocol.If a centralized learner exists, the task could be mapped to an MDP, whose centralized Q-function could be expressed asTeam Q-Learning (Team-QL) , i = 1, …, N is the state transition function Pt that defines the probability of observing s′ after agent δ(t) executes, at time t, action a from its local action repertoire Aδ(t).R:S × ∪Ai × S → ℝ is the shared scalar reward signal Rt received by all agents after executing a local a action from Aδ(t).δ:ℝ → {1, …, N} is the turn function implementing the Round-Robin cycle of agent calling for action execution. δ(t) gives the index of the agent allowed to execute an action at time t (times are relative to the start of the episode). This function is cyclic, ∀t ∈ ℕ;δ(t) = δ(t + N), therefore, agents are always visired in the same order, so that without loss of generality, we can denote i + 1 the agent that will be visited after i.The state value function Vπ gives the value of being in state s for agent i ∈ {1, …, N} under joint policy π, and it is the expected accumulated discounted rewards following the joint policy π. The Bellman equation for a joint policy π in a C-RR-SG isEπ represents the expectation from time t onwards, following joint policy π. The optimal state value V* is given by:π can be expressed asi isThe straightforward approach to learn the optimal policy for a C-RR-SG is applying the single-agent Q-Learning update:Proposition 1Q* if each agent fulfills the conditions for convergence of the single agent Q-learningA centralized single-agent Q-Learning training of a C-RR-SG by the rule of .Proof. If we are able to map the C-RR-SG into an MDP, then the proof of the proposition follows from the convergence of single agent Q-learning. The map is as follows: The MDP state space S is the same of the C-RR-SG, since it is defined by a set of global variables visible by all agents. The MDP set of actions is the union of the local sets of actions P = P, s′), that is, actions are sequenced according to the turn function. The MDP reward function is built by R(s′) = R, s′), that is, we record the reward at the arriving state. The C-RR-SG turn function removes the uncertainty and concurrency about the joint agent actions, therefore the MDP policy can be stated as π(st) = πδ(t)(st), that is, we apply at each time the local policy of the agent selected by the turn function. The MDP Q-table is built by collapsing all the agent Q-tables into a monolithic one by applying the turn function, i.e. Qt = Q). Single-agent Q-Learning converges to optimal Q* values under two conditions . Null action ∅ is defined as the action that does not perform any state transition: ∀s;P = 1.where joint-action discounted rewards of a joint-action a = , ai ∈ Ai, are approximated by the sequential execution of agents’ actions following RR scheduling δ:P ≃ P, aδ(2)…aδ(N), s′)R ≃ R, aδ(2)…aδ(N), s′),The transition probabilities and P, aδ(2)…aδ(N), s′) and R, aδ(2)…aδ(N), s′) denote the probability of reaching state s′ after executing actions specified in a in the sequence established by δ, and the expected reward of this transition, respectively. Under this assumption, Q-values for joint-actions can be approximated by composition of the transition and reward functions of local actions. The state-action value for composed joint-actions can be defined asa = [a1a2…aN], ai ∈ Aδ(i). Note that rewards inside a joint-action are weighted as in the C-RR-SG.where Greedy selection of optimal joint policy The combinatorial optimization problem of chosing the best ordering of the local actions has been tackled as a greedy variable selection process  with is the greedy solution to the following minimization problem:Generalizing this process, agent For completeness, terminal states must be taken care of, because we are composing cycles of actions. If a terminal state is reached, there will not exist transition probability information from this state onwards. We can address this potential implementation issue by leaving unaltered the entries in the incoming messages for which no transition probability is available, so that the remaining agents will perform no action at all.Proposition 3The action selection policy defined by Eq (8).Proof. The transition and reward functions are approximated by πD(s) ≃ π*(s).T∪G,Ae,P,R >.This approximation has been tested experimentally in environments with delayed rewards, with positive results. Unlike the technique presented in , where tN time-steps to perform N local actions instead of a single joint-action, and the execution time will grow linearly to the number of agents. Nevertheless, this is neglectable in the case of overconstrained environments, where learning in an effective manner Ae(s) is critical.An additional advantage of D-RR-QL is that it allows us to use MSAV in a multi-agent environments. Since we assume that a joint-action is approximately the same as the equivalent sequence of local actions, agents can veto actions on their local action space. Agents take actions one by one during the cycle of execution and thus have no interference from the rest of agents. Of course, this requires taking Modular State-Action Vetoes). Let the state be defined i-th Veto-Module, and let the joint-action space be defined A = Aj × A−, where Aj is j-th agent’s local action space, and A− = A1 × Aj−1 × Aj + 1 × … × AN. Whenever an agent vetoes a state-action pair in its own state-action subspace, it is effectively vetoing ∣SX × A− ∣ state-action pairs in the joint state-action space. The multi-agent Safe-MSAV architecture can be expected to avoid UTS much faster than any other exploration strategy operating on the complete state space S.The speed boost introduced by the use of MSAV in such multi-agent environments is even bigger than in the single-agent case  and the environment is reset to the initial setting  for the first robot).The relative positions of the Self-position in absolute coordinates. Each robot perceives its own absolute position.Four flag variables indicating whether there is an obstacle  in each of the four possible directions a robot can move.Benchmark algorithms  only use robot relative positions because the remaining state variables are a linear function of them and these algorithms cannot benefit from the additional information provided by the extended set of variables. For instance, it is of no use to include the obstacle flag variables because they are a function of the relative positions of the robots and thus, can only take one value for any given set of relative positions. In order to best learn how to avoid breaking a physical constraint corresponding to an UTS, we used three Veto-modules within each agent’s MSAV policiy, each of which in charge of a specific physical constraint:Overstretching the hose. This module receives a negative reward every time a hose segment is overstretched, and its local state space encompasses the relative positions of the robots preceding and following the robot which executed the action that led to over-stretching its hose segment.Collisions. Every time a robot takes an action that makes it collide with any other, this module receives a negative reward. The local state space is composed by the obstacle flag variables.Simulation bounds. This last module uses only its robot’s absolute position and receives a negative reward every time a robot gets out of the simulation grid.Each of the four generated negative rewards was fed to a specific module learning the veto of specific undesired state-actions using the value of the relevant state variables [Linked Multi-Component Robotic Systems (L-MCRS) are over-constrained systems which can benefit from the use of D-RR-QL with MSAV. The hose transportation task is a paradigmatic application of L-MCRS: A set of n robots . This si setting . The sysariables .ne. D-RR-QL with MSAV was run using the Safe Soft-Max policy ϵ − greedy policies but the experiments yielded slightly worse results and thus, will not be reported here. Initially, the temperature parameter was set τ = 10 and decreased at the end of each episode with △τ = −10/ne. Starting from episode number 500, the learnt policy was evaluated using greedy action selection each 500 episodes. This is the reason why data plots start from episode 500 instead of episode 0. Each experiment was run 3 times and the results were averaged. The performance was measured using two different criteria: (a) the accumulated discounted rewards, which is the typical performance measurement in the RL field, and (b) the number of cells reached with the tip of the hose, which allows us to intuitively assess how effective the exploration of the state-space is.Because the amount of episodes and the degree of exploration is key to this particular kind of tasks, experiments were run with different number of episodes http://www.ehu.es/ccwintco/index.php?title=D-RR-QL.We implemented ourselves these experiments in C/C++. The source code used in our experiments and can be accesed publicly at: ne = {104,2 ⋅ 104}. Figs even reaching the goal cell a single time.In this first experiment, we tested two different values of Figs ne = 5 ⋅ 105 and ne = 106). We only report the number of reached cells with the tip of the hose, because the accumulated discounted rewards are similar to those obtained in the previous experiment, and offer no farther information. Both Soft-Max and ϵ − greedy policies were tested and, in this case, slightly better results were obtained by using ϵ − greedy. Initially, ϵ = 1 and every episode it was decreased with △ϵ = −1/ne. This is paradoxical considering the better results obtained by Soft-Max policies in the previous experiment. Our educated guess is that this is due to using the same decay schedule for the two exploration parameters ϵ and τ. While they have a similar function, ϵ is the probability of exploring each time-step and τ shapes the probability distribution of the action selection algorithm. Moreover, as the Soft-Max algorithm depends on the magnitude on the actual estimations of the value function, the efficiency of the exploration algorithm needs not improve linearly to the number of episodes allowed to the agents.In order to get a clearer idea of how far the benchmark algorithms are from the goal of learning a greedy policy that reaches the goal position, we ran a second set of experiments where agents were allowed much more time for exploration of the environment  policies which improve performance in over-constrained environments. First, we have presented agent interaction model as a Cooperative Round Robin Stochastic Game (C-RR-SG), in which agents select and take actions following some predefined order. A communication-free distributed implementation called Distributed Round Robin Q-learning (D-RR-QL) has been proposed, providing a proof of its convergence to the optimal policy on a C-RR-SG. We have also presented a message-based procedure to obtain the optimal policy for the original cooperative stochastic game (C-SG) from the local policies learned by D-RR-QL. We give a consensus-based implementation for the decentralized determination of RR turns that may allow fully distributed implementation of the D-RR-QL.O(1) messages each step if communications are used, 0 messages otherwise) and to obtain the global policy deciding a joint-action (each action requires O(N) messages) are minimal when compared to coordinated multi-agent approaches such as Coordinated-QL. As a line of future work, we need to study the equivalence between C-SG and C-RR-SG to determine the degree of approximation that can be obtained when there is no C-RR-SG equivalent to a C-SG. We also plan to investigate more general ways to learn how to avoid reaching undesirable terminal states. On the other hand, the degree of abstraction used in our experiments limits the applicability to real robot control problems. This approach can be directly applied in real environment as long as low-level controllers and sensors provide this kind of abstract actions and states, but the coarseness of the state-action space representation limits the optimality of the learned policies. Therefore, we plan to use continuous state-action spaces in our future research.Computational experiments show that D-RR-QL using MSAV policies can provide a valid joint-action policy approximating the optimal policy faster than D-QL, Team-QL, and Coordinated-RL in over-constrained systems. Furthermore, the D-RR-QL convergence is not limited to deterministic MDPs, it can also cope with stochastic environments. Communication requirements to learn local Q-values (


On top of the q-group index we introduce PEANUT, a highly parallel GPU-based read mapper. PEANUT provides the possibility to output both the best hits or all hits of a read. Our benchmarks show that PEANUT outperforms other state-of-the-art read mappers in terms of speed while maintaining or slightly increasing precision, recall and sensitivity.We present the  A read mapping algorithm has to be error tolerant because of mutations and sequencing errors. The optimal solution to this problem is to calculate alignments of each read against the reference. Since this would incur an infeasible running time , various filtering methods and approximations have been developed. These can be roughly classified into methods based on backward search using the Burrows–Wheeler transform  data analysis projects, e.g., for variant calling or gene expression analysis, is mapping the obtained DNA reads to a known reference . The orm BWT; , e.g., Borm BWT;  or Bowtiorm BWT; , and metRazerS 3 . Both teAmong the best alignments of a read, it is sometimes not obvious which one represents the true origin of the read. In the following, the candidate origins of a read reported by a read mapper are called hits. q-gram index on a GPU is not a common choice because of its large size. Therefore, to the best of our knowledge, q-gram index based mappers so far only use the GPU for calculating the alignments and keep the index on the CPU, e.g., Saruman  for read mapping has become popular and GPU-based BWT-read-mappers appeared, e.g., SOAP3 , SOAP3-d Saruman  and Next Saruman .q-group index, a novel data structure for read mapping which is a variant of the classical q-gram index with a particularly small memory footprint. The q-group index comes with efficient parallel algorithms for building and querying, targeted towards modern GPUs. To the best of our knowledge, the q-group index is the first feasible implementation of q-gram index functionality on the GPU. On top of the q-group index we present PEANUT , a GPU-based massively parallel read mapper. PEANUT provides both an all-mapping and a best-mapping mode and is the first GPU-based all-mapper. With both a recent and a four years old NVIDIA™ Geforce GPU, we show that PEANUT outperforms other state of the art best-mappers and all-mappers. For all-mapping, PEANUT is 4–10 times faster. PEANUT shows a slightly higher precision and recall than other best-mappers and an improved sensitivity compared to other all-mappers at default parameters.Here, we present the q-group index to maximize parallel GPU usage (“Designing for efficient GPU usage”). Then, we describe the q-group index data structure (“The q-group index”) and present the PEANUT approach of read mapping with the q-group index . The “Results” section shows benchmark results on speed, precision, recall and sensitivity of PEANUT. A brief discussion concludes the paper.This article is structured as follows. We first discuss the GPU architecture and its implications for designing the thread block size it can be controlled how threads are distributed among the SMs. One thread block is executed on one SM and continues resident until all threads in the block are completed. Once a thread block has finished, another will be scheduled to the SM if any blocks are left. An SM can execute 32 threads in parallel (restricting the thread block size to be a multiple of 32); such a group of threads is called a warp or wavefront. At any time, each of these threads has to execute the same instruction in the code, but may do so on different data, a concept which is called single instruction, multiple threads (SIMT). Hence, conditionals with diverging branches should be avoided, since threads taking an if-branch have to wait for threads taking the corresponding else-branch to finish and vice versa. All SMs may access a slow common global memory (about ≤3 GB on most of today’s GPUs) in addition to their fast on-chip cache and memory. While the size of the fast cache is extremely limited, accesses to the global memory are slow and should be minimized. However, the memory latency can be reduced by coalescing the access, i.e., letting threads in a warp access contiguous memory addresses, such that the same memory transaction can serve many threads. In addition, an SM can execute a different warp while waiting on a transaction to finish, thereby hiding the latency. For the latter, threads should minimize their register usage such that the number of warps that can reside on an SM is maximized. Finally, data transfers from the main system memory to the GPU global memory are comparatively slow. Hence it is advisable to minimize them as well.We use the terminology of NVIDIA™, while the general concepts are also applicable to competitors like AMD™. A GPU is partitioned into streaming multiprocessors (SMs), each of which has its own on-chip memory, cache and processing cores. By adjusting the parallel prefix scans, a special case of which is the computation of a cumulative sum, which at first appears to be a serial process, or filtering an index set. Parallel prefix scans are used to solve these problems in a data parallel way with a minimum amount of branching, thus nicely fitting the above considerations; A useful programming pattern that is used extensively in the algorithms presented in this work are q-group index presented in this work is designed with above considerations in mind: The data structure supports simple access patterns that minimize the register usage. This allows to hide latency caused by not coalesced memory accesses. We evaluate this in “GPU resource usage”. The parallel algorithms presented in this work further minimize the data transfers necessary between their steps to constant amounts of data . Where appropriate, we discuss architecture specific considerations in the text.The q-gram index of a text T stores, for each string of length q over the alphabet, at which positions in T the q-gram occurs and allows retrieving these positions in constant time per position. It is commonly implemented via two arrays that we call the address table A and the position table P.A classical DNA Q-grams are encoded as machine words of appropriate size with two subsequent bits encoding one genomic letter . Unknown nucleotides  are converted randomly to A, C, G or T, and larger subsequences of Ns are omitted from the index. Hence, a q-gram needs 2q bits in hardware and is represented (encoded) as a number g∈{0, …, 4q−1}. The address table provides for each (encoded) q-gram g a starting index A[g] that points into the position table such that P[A[g]], P[A[g] + 1], …, P[A[g + 1]−1] are the occurrence positions of g.q-gram length q entails a tradeoff between specificity of the q-grams and the size of the data structure. Array A needs 4q integers and thus grows exponentially with q, while array P needs |T| integers, independently of q. Larger values of q lead to fewer hits per q-gram that need to be validated or rejected in later stages. Further, the choice of q determines the sensitivity or error-tolerance of the search via the pigeonhole principle (q-gram lemma): only if there are e < (n + 1)/q−1 errors, we can guarantee that a q-gram match exists.Deciding about the q-group index is to have the same functionality as the q-gram index , but with a smaller memory footprint for large q. This is achieved by introducing additional layers in the data structure. In the following, we always consider a q-gram as its numeric representation g∈{0, …, 4q−1}.The idea of the qq-grams into groups of size w, where w is the GPU word size . The q-gram with the number g is assigned to group number ⌊g/w⌋. Thus the ith group is the set iG = {g|⌊g/w⌋ = i} of w consecutive q-grams according to their numeric order. The set of all q-groups is ijg for the jth q-gram in iG.We divide all 4q and text T, the q-group index is a tuple of arrays I consists of w bits each , and bit j of I[i] indicates whether ijg occurs at all as a substring in the text, i.e., For a given O corresponds to the position table P of a regular q-gram index: it is the concatenation of all occurrence positions of each q-gram in sorted numeric q-gram order. To find where the positions of a particular q-gram g begin in O, we first determine the group index i and the q-gram number j within the group, such that g = ijg. With the bit pattern of I[i], we determine whether ijq occurs in T. If not, there is nothing else to do. If yes, i.e., I[i]j = 1, we determine the j′ such that bit j is the j′th one-bit in I[i].The array S contains, for each group i, an index into another address array S′, such that S′[S[i] + j′] is the starting index in O where the positions of ijg can be found; see The address array q-gram index, access is in constant time per position: For a given q-gram g, to determine  such that g = ijg, we simply compute i = ⌊g/w⌋ and j = g−wi = jmodw. To compute j′, i.e., find how many one-bits occur up to bit j in I[i], we use the population count instruction with a bit mask: x) returns the number of 1-bits in x.Similar to a plain I is created from the q-grams of the text (line 2). Then, S is calculated as the cumulative sum over the population counts of I (line 6). Next, the number of occurrences for each q-gram is calculated (line 10) and S′ is created as the cumulative sum over these counts (line 14). Finally, the q-gram positions are written into the appropriate intervals of O (line 16).Algorithm 1 shows how the index is built. The outline of of the algorithm is as follows. First, 1https://www.khronos.org/opencl kernels. The cumulative sums are implemented with parallelized prefix scan operations (see “Designing for efficient GPU usage”). Importantly, the algorithm needs hardly any branching (hence maximizing concurrency) and makes use of coalescence along the reads in order to minimize memory latency. All major data structures are kept in GPU memory.Each step is implemented on the GPU with parallel OpenCLq-group index, we note that both I and S consist of ⌈4q/w⌉ words, S′ contains an index for each occurring q-gram and hence of min{4q, |T|} words in the worst case, and O is a permutation of text positions consisting of |T| words. Thus the q-group index needs up to 2/w⋅4q + min{4q, |T|} + |T| words, compared to a conventional q-gram index with 4q + |T| words. To evaluate how the worst case size of the q-group index behaves compared to the size of the q-gram index, we consider the ratio K between the possible q-grams and the text size, i.e., 4q = K|T|.To determine the worst case size of the q⪅|T|, the conventional q-gram index has a small advantage because each q-gram occurs (even multiple times). In fact, assuming w = 32, if K ≤ 1, the size ratio between q-group index and q-gram index is K = 1 (or |T| = 4q), this means a small size disadvantage of 3% for the q-group index.If 4q becomes larger for fixed text size (such that q-grams become sparse), the q-group index saves memory, up to a factor of 16: For K > 1 the size ratio is (K/16 + 1 + 1)|T|/((1 + K)|T|) = (2 + K/16)/(1 + K), which tends to 1/16 for large K. The break-even point is reached for K = 16/15.If, on the other hand, q = 16 because a bit-encoded DNA 16-mer just fills a 32-bit word and q = 16 offers reasonable error tolerance and high specificity. In this regime, with current GPU memory size, we use |T| = 108 (processing 100 million nucleotides at a time), so the size ratio is K = 42.95, and the q-group index needs only 10% of the memory of the conventional q-gram index. K.In practice, we use q-group index further by sampling every second  position of S and adding another population count and addition instruction to the code for odd positions.Without noticable increase in access time, we can reduce the memory usage of the q between each read and each reference  and only computes alignment scores where such matches are found. In total, the PEANUT algorithm for read mapping consists of the three steps (1) filtration, (2) validation and (3) postprocessing. The first two steps, filtration and validation, are handled on the GPU, while postprocessing is computed on the CPU.PEANUT uses the filtration plus validation approach: It first quickly detects exact matches of a given length The steps are conducted on a stream of reads. Reads are collected until buffers of configurable size are saturated. Then, any computation is done in parallel for all buffered reads see .q-group index. Next, the potential hits are validated using a variant of Myers’ bit-parallel alignment algorithm  between the reference and the reads.The filtration step aims to yield a set of potential hits, e.g., associations between a reference position and a read. For this, we seek for matching q-group index is created on the GPU on the fly (see Algorithm 1). If the index was built over the reference, either repetetive large data transfers or on-line rebuilding of the index would be necessary for each set of buffered reads and each chromosome. Hence—inspired by RazerS 3  that returns, for a q-gram g, an index pair  such that the occurrence positions in the indexed text are all O[k] with kstart ≤ k < kend, where O is from the q-group index built with Algorithm 1. Given the q-group index , the function Indexpair(g) is implemented as follows. Let ≔Group-And-Bit(g) and j′≔Grouprank. Then kstart = S′[S[i] + j′] and kend = S′[S[i] + j′ + 1]. We assume that there is a function Indexpair. In the following, only positions with at least one hit are considered (line 5). The cumulative sum of the counts generates an interval for each position, that determines where its hits are stored in the output array of the algorithm (line 6). Finally the occurrences for each reference q-gram are translated into hits that are stored in the corresponding interval of the output array (loop in line 8). We translate the position inside the index of concatenated reads into a read number (line 12) and a “hit diagonal” that denotes the putative start of the read in the reference  uses parallelized prefix scan operations (see “Designing for efficient GPU usage”). All data structures reside in GPU memory; between the steps, at most constant amounts of data have to be transferred between host and GPU .Again, each step of Algorithm 2 is implemented on the GPU with parallel OpenCL kernels. The filtering of P of reference positions to investigate and the reference sequence are retrieved from a precomputed HDF5-based reference index.2http://www.hdfgroup.org/HDF5.Hierarchical data format version 5,  First, this speeds up access to the reference. Second, we omit exceedingly frequent q-grams from the set P, as these only yield uninformative hits when using q-gram index based algorithms. Finally, P is sorted in numerical order of the q-grams. This increases the memory coalescence when accessing the q-group index, since subsequent threads will have a higher probability to access the same region in the index and hence the same memory bank in the global GPU memory.The set The validation step takes the potential hits of the filtration step and calculates the edit distance  between a read and the reference sequence at its putative mapping position. If the edit distance is small enough, the hit is considered to be correct and will be postprocessed in the next step.E between reference sequence and read, which contains one column for each reference base and one row for each read base. The value ijE is the minimal edit distance between the read prefix of length i and any substring of the reference that ends at position j. Interpreted as a graph with a node for each matrix entry, a path in E from the top to the bottom row denotes a semi-global alignment between read and reference. Myers’ algorithm calculates the edit matrix column-wise. The state of the current column is stored in bit vectors. A transition from one column to the next is achieved via a constant amount of bit-parallel operations on the bit vectors. In iteration j, the minimal distance between the read and any substring of the reference that ends at position j can be retrieved. If the accepted error rate is limited, only a part of the edit matrix is needed to calculate the optimal edit distance. Our implementation of the algorithm follows a banded version that calculates only the relevant diagonal band of the edit matrix (w (currently 32 bits). Thereby it provides a complexity of r| being the read length. While the reduction to the diagonal band restricts the maximum insertion or deletion size in a single alignment, mismatches are not affected. Hence, the procedure allows to discover partial matches of the read, as needed for split read mapping. Large indels can be rescued later when calculating the actual alignment if a sufficiently large portion of the read aligns in this step.The edit distance is calculated with Myers’ bit-parallel algorithm  that simt matrix . The imppercent identity is obtained which we define here in compliance with RazerS 3 as (|r|−k)/|r|, where |r| is the read length and k is the edit distance. Hits with a percent identity less than a given threshold are discarded. The default for this threshold is 80 percent which provides a decent sensitivity in our benchmarks (see “Results”). Decreasing it has moderate impact on performance, since more hits have to be postprocessed and written to disk.Similar to q-grams), sorts the remaining hits by percent identity, pairs mates in case of paired-end alignment, estimates a mapping quality and calculates the actual alignment of each hit. For the latter, either local or semi-global alignments can be chosen. In contrast to the previous steps, postprocessing is done on the CPU. This allows us to postprocess the hits in parallel to filtration and validation.The postprocessing of a read removes duplicate hits  that the hit position p is not the true origin of the read r in the reference. p|r) in a Bayesian way as P being the set of all reference positions and approximate the likelihood Pr(r|p) of a read r to be sampled from position p  as product of the miscall probabilities of mismatching bases as obtained from the sequencer. In practice, Pr(p|r) is approximated roughly using the best and second best hit  we can approximate the posterior probability Pr(p|r) as P′ being the validated hit positions of the read r. The PHRED-scaled mapping quality is then obtained as min{−10log10(1−Pr(p|r)), 60}. Per default, we cap the mapping quality at 60 and force it to 0 in case of ambiguous hits  in order to generate values comparable to other read mappers like BWA  from false positives is needed. PEANUT reports alignments in the SAM  format, best hit . PEANUT like BWA . This isWe evaluate PEANUT in terms of its efficiency of GPU resource usage, its accuracy and its speed. We also evaluate the ability of the mapping quality measure defined in “Postprocessing” to separate true hits from others.occupancy, that is the fraction of active warps among the maximum number of warps on an SM. The more active warps exist on an SM, the higher is the chance that latency can be hidden by executing another warp. q-group index (create_queries_index) and the filtration step (filter_reference), it even reaches 1.0 which illustrates the benefit of the q-group index being tailored towards the GPU architecture.In order to maximize utilization of the GPU hardware, idle cores have to be avoided. The two most important reasons for idle cores are branching and memory latency (see “Designing for efficient GPU usage”). The latter can be hidden if the SM is able to execute a different warp while waiting on the transaction. The capability to do so can be measured with the q-group index) to detect all alignments up to a given error rate. For this, we use the Rabema , such that using 16-grams appears to be a reasonable default choice in practice.In general, we advise to set the percent identity threshold to be slightly more permissive than the expected error rate. This is because the replacement of We compare the PEANUT algorithm with other state of the art read mapping algorithms in terms of run time and accuracy. The evaluation is conducted on 4 datasets:1.5 million simulated Illumina HiSeq 2000 reads;2.5 million real Illumina HiSeq 2000 reads from the human exome;3.10 million real paired-end Illumina HiSeq 2000 reads from the human exome;4.50 million real paired-end Illumina HiSeq 2000 reads from the whole human genome.5http://ftp.ensembl.org/pub/release-74/fasta/homo_sapiens/dna/ version 37 with Mason (see “Sensitivity”). The read length is set to 100 and all other parameters of Mason are left at their default values, such that reads with a typical error profile and mutation rate are generated. The second and third datasets are generated from real paired-end exome sequencing reads6http://www.ebi.ac.uk/ena/data/view/ERR281333, 07/2014.Available at the European Nucleotide Archive under   were created from the ENSEMBL human reference genome07/2014.  of lengtparallel (the time for merging the resulting output was not included into the run time). NextGenMap was used in GPU mode, such that it makes maximum use of the available hardware. All read mappers were configured to output alignments in SAM format  with an NVIDIA Geforce™ 780 GPU  and a 7200 rpm hard disk. We evaluated two modes of PEANUT. First, PEANUT was configured to find the best stratum of semi-global alignments (best-stratum mode) for each read. Second, PEANUT was configured to find all semi-global alignments  for each read. For comparison, we benchmarked the newest generation of BWA , no working installations of SOAP3 , SOAP3-dIn the following, we distinguish between all-mappers and best-mappers . Best-mappers only strive to find the single origin of a read on the reference sequence of a single organism. All-mappers provide all alignments of a read down to a given error rate. Hence, all-mapping is computationally more intensive.Run time performance is measured three times as the overall wall clock time for processing a dataset on the test system. 9http://www.bioplanet.com/gcat, 08/2014. benchmark provides an alternative, less formal approach to measure the accuracy of a mapper by testing whether a mapped read is within 5 base pairs of its known origin in given datasets simulated with DWGSIM.10https://github.com/nh13/DWGSIM, 08/2014. For comparison, we provide GCAT results in 11http://www.bioplanet.com/gcat/reports/3499-acxvkaiakl/alignment/100bp-se-small-indel/peanut/compare-26-35, 08/2014. (2) 100bp-se-large-indel,12http://www.bioplanet.com/gcat/reports/3522-twmulhzdnt/alignment/100bp-se-large-indel/peanut/compare-28-32, 08/2014. (3) 100bp-pe-small-indel13http://www.bioplanet.com/gcat/reports/3500-rzzjxccatd/alignment/100bp-pe-small-indel/peanut/compare-23-18, 08/2014. and (4) 100bp-pe-large-indel,14http://www.bioplanet.com/gcat/reports/3498-dqxwpnmjyn/alignment/100bp-pe-large-indel/peanut/compare-87-25, 08/2014. consisting of about 12 million 100 basepair reads each. The comparison is limited to those of our considered read mappers for which public results are available in GCAT . The full reports are available via the GCAT web service under the URLs provided in the footnotes.The accuracy of the obtained alignments is assessed using Rabema . For best-mappers, using the simulated dataset 1, we measure precision and recall with Rabema as defined by All-mappers are compared by their ability to find all alignments of a given error rate. Here, we again use the sensitivity provided by Rabema (see “Sensitivity”). This involves creating a gold standard with RazerS 3 configured to full sensitivity, which is computationally expensive. We therefore perform this on only 1,000 reads simulated with the same parameters as dataset 1. The gold standard is calculated for an error rate of at most 15%. While PEANUT requires at least 2.5 GB of GPU memory for filtration and validation, it is not restricted to running on high-end GPU models like the Geforce 780 used above: q, the corresponding probability λ and C of the approximation (see “Postprocessing”) can be configured to different values to achieve a closer fit for a particular reference or dataset. Despite that, the defaults provide a reasonable way to distinguish between true and false positives without the computational overhead needed for calculating concrete alignments of many suboptimal hits (see “Postprocessing”).Finally, we evaluate the mapping qualities provided by PEANUT. For each PHRED-scaled mapping quality q-group index, a novel data structure for read mapping, along with parallel algorithms for index building and querying. The algorithms fit nicely to the GPU architecture by using a combination of element-wise and prefix scan operations over large arrays and requiring hardly any data transfer between the host and the GPU. We showed that the q-group index needs significantly less memory in practical scenarios than a conventional q-gram index while maintaining constant access time. The q-group index has been adopted by NVIDIA™ and is provided as part of the NVBIO library.15http://nvlabs.github.io/nvbio/We presented the q-group index we implemented the read mapper PEANUT. The q-group index enables the mapper to be the first implementation of the filtration and validation approach that works completely on the GPU. The benefit of this is illustrated by PEANUT showing supreme speed over state of the art read mappers like BWA, Bowtie 2, RazerS 3, CUSHAW3 and MrFast, as well as the GPU based read mappers NextGenMap and CUSHAW2-GPU. This holds for both the high-end GPU NVIDIA™ Geforce 780 and a four years old NVIDIA™ Geforce 580. PEANUT is 4–10 times faster than other all-mappers. Further, it is faster than all other best mappers; in particular, it is 2 times faster than the fastest best-mapper on the most realistic dataset. The speed improvements do not come at the cost of mapping quality. In fact, PEANUT is even slightly more sensitive than other all-mappers at default parameters and provides an at least comparable precision and recall compared to other best-mappers in our Rabema-based benchmarks.On top of the 16http://www.python.org software package. Filtration and validation were implemented in OpenCL, using the PyOpenCL package (http://peanut.readthedocs.org, where we further provide a Snakemake (PEANUT is distributed under the MIT license as an open source Python package  and post package . Documennakemake  workflow


Most psychological research on Bayesian reasoning since the 1970s has used a type of problem that tests a certain kind of statistical reasoning performance. The subject is given statistical facts within a hypothetical scenario. Those facts include a base-rate statistic and one or two diagnostic probabilities. The subject is meant to use that information to arrive at a “posterior” probability estimate. For instance, in one well-known problem Eddy,  the subjThe probability of breast cancer is 1% for a woman at age forty who participates in routine screening. If a woman has breast cancer, the probability is 80% that she will get a positive mammography. If a woman does not have breast cancer, the probability is 9.6% that she will also get a positive mammography. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? __ %.H as the focal hypothesis, ¬H as the mutually-exclusive hypothesis, and D as datum: P(H), the prior (often equated with the base-rate) probability of the hypothesis; P(D|H), the true-positive rate; and P(D|¬H), the false-positive rate. In the mammography problem, P(H) = 0.01, P(D|H) = 0.80, and P(D|¬H) = 0.096. Furthermore, P(¬H) = 1 – P(H) = 0.99. The estimate queried is P(H|D).The information in such problems can be mapped onto common expressions that use Bayes' theorem states:Thus, it yields a posterior probability of 0.078 in the mammography problem. Yet even the majority of physicians who were queried by Eddy  gave estWell-established findings such as these have supported the view that expert and naïve subjects alike are non-Bayesian , which they are asked to estimate, with P(D|H), which is provided  = 0.80. The inverse fallacy can also explain patterns of deviation from Bayes' theorem in tasks that hold constant base rates for alternative hypotheses , and researchers typically do not collect multiple “prior” and “posterior”  probability assessments.A remarkable feature of the standard approach to studying Bayesian reasoning is its inability to reveal how people P(H), which the subject updates in light of “new” evidence, D. It is somewhat ironic that advocates of base-rate neglect have not noted  that, if people ignore base rates, it may be unwise to assume they represent the subject's prior probability. Would that not imply that the subject ignores his or her own prior probability?It is instead conveniently assumed that the base rate represents the subject's prior belief, P(H), is in fact a conditional probability corresponding to one's personal probability of H, given all that they know prior to learning D , b = f (H ∩ ¬ D), c = f (¬ H∩ D), and d = f (¬H ∩ ¬D). Going from left to right, the four boxes in the lowest level of the frequency tree in Figure a–d, which have received much attention in the causal induction literature /(a + b + c + d), a/(a + b), and c/(c + d). That is, frequencies a–c support the easy computation of a/(a + c). However, those improvements in performance, which pertain to static coherence constraints  ranged from 0 to 1 over seven probability levels across the problems. Subjects exhibited a form of conservatism , causing an underweighting of relevant information.For example, Williams and Mandel  presented Mandel  found thThe issues I have raised, non-exhaustive as they are, draw attention to some important problems with the conventional approach to studying Bayesian reasoning in psychology that has been dominant since the 1970s. Rather than fostering pessimism, I hope my comments illustrate that there are good opportunities for future work to advance our understanding of how people revise or update their beliefs.The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


One fundamental yet longstanding issue in materials science is how local inelasticity arises within an amorphous structure before yielding occurs. Although many possible scenarios were postulated or predicted by theories and simulations,however, direct experimental evidence has been lacking today due to the lack of a sensitive way to detect nano-scale inelasticity. Through the carefully designed microcompression method as coupled with the state-of-art nano-scale electric resistance measurement, we here unfold a stochastic inelastic deformation process in a Zr-based metallic glass, which takes place via the recurrence of two types of short-lived inelastic events causing structural damage and recovery, respectively, prior to yielding. Our current findings reveal that these stochastic events not only self-organize into sub-critical events due to elastic coupling, but also compete with each other in a way that enables the whole amorphous structure to self-heal as well as to sustain local damage. Amorphous materials are ubiquitous in our everyday lifeOver the past decades, it is the long-sought issue with regard to the origin of flow in amorphous solids that has been attracting tremendous research interest of materials scientists and physicists234567891011121314153481011Here, a coupled nano-scale electro-mechanical system was used to tackle this problem. As seen in the later text, this system is proven to be extremely sensitive in resolving the structural signals that result from inelastic events occurring during the apparent 'elastic' deformation of a metallic glass. As compared to a conventional mechanical test, the nano-mechanical module of this system is based on the recently developed Hysitron TI950 nanoindentation platform, which possesses the resolution of ~0.1 nm in displacement and ~1 μN in force; in addition, the coupled nano-electric module is based on the state-of-art Nano-Electric-Contact-Resistance (NanoECR) technique, which possesses the resolution of 10–100 pA in current and 0.1 mA in voltage See . The cou55Cu30Ni5Al10 (in at. %). Its structural amorphousness was confirmed using the X-ray diffraction method and high-resolution transmission electron microscopy  and electric current (I) data, as a function of time (t), measured simultaneously from the microcompression test. It should be emphasized here that, due to the surface roughness effect and the settling of the nanoindenter on the micropillar, there is a period of 1–2 seconds before an electric Ohm's contact could be established  and the electric current I. In the above relation, Δ represents an incremental change of a physical quantity and ε = 0h/H, where 0H denotes the initial height of a micropillar. Here it should be noted that, due to the ultrahigh sensitivity of our testing system, the noise-to-signal ratio for the calculated Δρ/ρ is very low and about 1 × 10−4  of the metallic glass can be extracted from the measured displacement (h) and current (I) data. Δρ/ρ and h versus time t as the micropillar was compressed to a peak stress of ~2000 MPa. Before reaching this peak stress, a few pop-in events can be witnessed on the h-t curve. Note that, for the sake of consistency, a displacement pop-in event is herein defined as a displacement jump greater than ~2 nm in size. In accord with the pop-in events, a few pronounced resistivity spikes can be also observed on the Δρ/ρ -t curve. As seen in the inset of 45Δρ/ρ| greater than 5 × 10−4, which is about fivefold of the maximum experimental error that could occur to the derived resistivity change. As such, we believe that these secondary spikes are the manifestation of real structural alteration caused by the local inelastic events in the metallic glass.With the above equation, the relative change in the electric resistivity  the structure factor, and μ(q) the pseudo-potential which can be replaced by a scattering matrix when the above model is applied to metallic glasses. In the 1980s, Kelton and SpaepenΔρ, of a metallic glass can be directly related to the variation in its atomic packing density Δξ through Δρ = CΔξ. Here C is the pre-factor that depends mainly on the structural factor S(q) of the metallic glass  with thermal annealing if α < 5 × 10−5 K−1 or otherwise increases with thermal annealing. On the other hand, based on the data from 13 types of metallic glasses with different glass transition temperatures gT, Kato and co-workersgαΤ ~ 8.24 × 10−3 for metallic glasses. As the gT of Zr-based metallic glasses are around 700 K, the corresponding α should be about 1.2 × 10−5 K−1. This suggests that the electric resistance of Zr-based metallic glasses should decrease with thermal annealing or structural ordering, which is consistent with the previous result reported by Bai and co-workersΔρ/ρ of the Zr-based metallic glass to structural disordering while the decrease to structural ordering. In other words, the recurrence of these p- and n-type spikes indicates that the atomic structure of the metallic-glass micropillar was repeatedly experiencing damage and recovery even though the applied stress was far below the yielding point.Physically, the resistivity, h increases with time t during loading [h-t curves, agreeing with the ‘elastic' character of the overall deformation in the metallic glass. In sharp contrast, the corresponding curves of Δρ/ρ versus t display many spike-like features. Compared to those prominent spikes at displacement pop-in, these spikes occurring in the ‘elastic' regime appear much shorter and last only for a short period of time. Due to the limit of our data acquisition rate (~103 points per second), the measured duration times dΔt for these secondary spikes range from ~1 to ~1.5 ms and appear to be independent of the spike heights and the applied stresses  and densely-packed (solid-like) atoms8910232425262729302933p for the net transition of local structural configurations along the stress direction at time t and shear stress τ, it can be proved that p satisfies the following dynamic equation in order to retain the total free energy of the system29ω is the attempt frequency; ΔG is the energy barrier and Ω is the activation volume of the local inelastic event. For a constant shear stress τ at load hold, we obtain: Rt is the average relaxation time of these inelastic events, which equals Rt/t goes to infinity, the ‘memory' of the MG about the stochastic inelastic events at t = 0 will die out while the probably p for the net transition along the stress direction will approach a steady-state value p(τ) = τΩ/kT. Note that Ω should be p(τ)f(Ω)dΩ, where f is the probability density function of the activation volume for an inelastic event. If the ‘liquid-like cell' model of Cohen and GrestLLRp is the volume fraction of the liquid-like cells, cΩ as the smallest activation volume. In such a case, the following cumulative probability can be obtained for the constant stress activation of the local inelastic events with their activation volumes greater than Ω: Assuming a probability With intΔt ahead of the event. Experimentally, intΔt can be measured as the time interval between two consecutive spike events of the same kind. intΔt obtained for the holding stresses of 296 MPa and 592 MPa, respectively. Consistent with our theory, the general trend of the waiting times intΔt clearly shows the distribution of an exponential decay. Meanwhile, as the holding stress is doubled, it appears that longer waiting times could be detected, which also agrees with our prediction. However, it needs to point out that this agreement is only in a qualitative manner. Rather than the strong linear stress dependence, the stress dependence of the accumulative probability, as seen from our data, is very weak. A similar behavior could be observed at other holding stresses. This weak stress dependence suggests that the simple mean-field anelastic model we proposed needs further development. One possible way of improvement is to include a term of elastic coupling into the mean-field modeling. In such a case, we can study how the variation of the stress field is coupled with the variation of the local properties, which may explain the weak stress dependence shown by our experimental data. The related theoretical work is under progress and will be discussed in our future work. Despite the weakness of the mean-field modeling, it is still worth mentioning that, with the mean-field model and dΔt~ 1 ms, we can estimate an average activation energy of ~0.5 eV for these local inelastic events, which agrees with the previous results obtained from nanoindentation29The Estimation of Activation Energy' in the To verify the above thinking, let us follow the recent analysis of Krisponeit et al.−3), is likely to be caused by the avalanches triggered through elastic coupling. Nevertheless, we envision that the latter mechanism due to the de-caging effect is also feasible for the high-magnitude spikes; however, the probability for its occurrence must be very low.Next, we would like to further address the following questions: in what cooperative way these chain-like inelastic events occur? And even if the occurrence of these events is stochastic in nature, are their outcomes, such as the structural relaxation and damage, also stochastic? In principle, these local inelastic events may occur in an isolated manner as pictured in the previous mean-field theories15Finally, let us discuss whether the outcome of the inelastic events, i.e. structural damage versus relaxation, is stochastic. As seen in 736To conclude, it is demonstrated in this Letter that, with the state-of-art NanoECR technique, the local inelastic events hidden, otherwise, in the overall elastic response of the metallic glass could be revealed. Due to the elastic coupling, these local inelastic events take place cooperatively, exhibiting a self-organized behavior as seen in their magnitude distribution. In contrast to the classic model, it is also shown that the effects of these inelastic events are twofold: it can either damage or heal the amorphous structure. Depending on the applied stress and stress rate, the competition of the local events can lead to overall damage or recovery of the amorphous structure. These intriguing and multi-scaled deformation characters as seen in the metallic glass are parallel to those already known phenomena observed for earthquakes, sand pile avalanches, dislocation slip dynamics and a variety of other dynamic systems36The Zr-based metallic-glass micropillars were prepared by FEI Quanta 3D 200 FIB/SEM dual beam system. Following the ion-milling method detailed in Ref.C.T.L. and Y.Y. designed the project. S.W. and Z.Y.L. carried out the experiments. X.L.F., B.A.S., Y.F.Y., Y.Y. and C.T.L. analyzed the data. Y.Y. and X.L.F. wrote the manuscript.Supplementary Information


K-means method is applied to segment 1,462 patients into twelve clusters. The average values of L, R, and F excluding monetary covered by national health insurance program are computed for each cluster. In addition, customer value matrix is used to analyze customer values of twelve clusters in terms of frequency and monetary. Customer relationship matrix considering length and recency is also applied to classify different types of customers from these twelve clusters. The results show that three clusters can be classified into loyal patients with L, R, and F values greater than the respective average L, R, and F values, while three clusters can be viewed as lost patients without any variable above the average values of L, R, and F. When different types of patients are identified, marketing strategies can be designed to meet different patients' needs.This study combines cluster analysis and LRFM  model in a pediatric dental clinic in Taiwan to analyze patients' values. A two-stage approach by self-organizing maps and  The medical care industry in Taiwan has become very competitive when the national health insurance (NHI) program with a mandatory, single-payer social health insurance system was launched by the government in Taiwan since March 1995 . Under sMarket segmentation is very useful to identify different customer needs and then to provide the management to customize the products or services to fulfill the needs . Data mi L, R, and F values compared with the overall average values of L, R, and F values, different types of patients, such as loyal patients, potential patients, and lost patients, can be identified. In addition, in order to evaluate customer values, customer value matrix proposed by Marcus [This study intends to combine data mining techniques  and LRFM model in a pediatric dental clinic for market segmentation. When all patients are grouped into clusters, the characteristics for each cluster are to be observed and described. By observing the average values ofy Marcus  by consiy Marcus  by takinThis paper is organized as follows.  K-means method are commonly used for cluster analysis.Huang et al.  stated tSOM, an unsupervised neural network method, uses clustering, visualization, and abstraction to solve problems and market screening –15. The K-means method, a nonhierarchical approach, is one of the very popular approaches to perform cluster analysis due to its simplicity of implementation and fast execution [ K-means method using Euclidean distance has two major steps. First, place the instances in the closest class (the assignment step). Second, recalculate class centroids from the instances assigned to the class (the reestimation step) [xecution . K-meanson step) , 17. K-means method is very sensitive to the choice of a starting point to partition the items into K initial clusters [ K-means method has been proposed and widely applied to improve the weakness [ K-means method to perform cluster analysis of the data set in the second stage.Self-organizing maps method has the advantages over other traditional cluster analysis methods but it does not provide measures for validation for the cluster analysis results . In contclusters , 19, 20.weakness , 20, 21.LRFM model was developed based on RFM model, which is a well-known method to analyze customer values for market segmentation , 22. TheThe traditional approach to use RFM model is to sort the customer data and then divide the data into five equal segments for each dimension of RFM . The topL) into consideration, where L is defined as the number of time periods (such as days) from the first purchase to the last purchase in the database.Reinartz and Kumar  addresse R is redefined as the number of days since the last visit when the first day of the specified time period is set to one. That is, R value belongs to the larger-is-better scenario, and larger R value indicates that the customer has visited the organization more recently, which is different from the traditional definition of recency. By further taking into account the strategic positioning of customer clusters, four major types of customers can be grouped in terms of RFM symbols, that is, lost customers with  R ↓F ↓M ↓ and R ↓F ↑M ↑, new customers with R ↑F ↓M ↓, loyal customers with R ↑F ↑M ↑, and promising customers with R ↑F ↓M ↑.Ha and Park  used aveL ↑R ↑F ↑M ↑, L ↓R ↑F ↑M ↑, L ↑R ↓F ↑M ↑, and L ↓R ↓F ↑M ↑), frequent customers , spender customers , and uncertain customers . Moreover, sixteen combinations of LRFM model can be classified into five major groups such as core customers, potential customers, lost customers, new customers, and resource-consumption customers. Specifically, core customers include L↑R↑F↑M↑, L↑R↑F↑M↓, and L↑R↑F↓M↑. Potential customers consist of L↑R↓F↑M↑, L↑R↓F↑M↓, and L↑R↓F↓M↑. Lost customers are composed of L↓R↓F↑M↑, L↓R↓F↑M↓, L↓R↓F↓M↑, and L↓R↓F↓M↓. New customers comprise L↓R↑F↓M↓, L↓R↑F↑M↓, L↓R↑F↓M↑, and L↓R↑F↓M↓. Finally, resource-consumption customers are L↑R↑F↓M↓ and L↑R↓F↓M↓.Chang and Tsay  based on F and M values, four quadrants as shown in F↑M↑), spender customers (F↓M↑), uncertain customers (F↓M↓), and frequent customers (F↑M↓). On the other hand, Chang and Tsay [ L and R to explain the characteristics of lost customer and loyalty by proposing customer relationship matrix depicted in L↑R↑), potential relationship (L↑R↓), lost relationship (L↓R↓), and establishing relationship (L↓R↑).Marcus  proposedand Tsay  used L aWhen different LRFM combinations are identified, customers can be classified into appropriate groups such as core customers, potential customers, lost customers, new customers, and resource-consumption customers. In addition, the customer value matrix enables the management to classify customers into best, spender, uncertain, and frequent customers in terms of frequency and monetary. Customer relationship matrix helps the management identify the characteristics of four different types of the relationships between the organization and the customers through length and recency. In other words, customer values can be evaluated by customer value matrix and customer relationship matrix. More importantly, different marketing strategies can be designed to meet different customer needs.This pediatric dental clinic started its operation since 17 September 1995. By definition, the patients must be less than 18 years old. This study collects the data set with 1,462 effective patients who visited this clinic with the total of 7,957 times from 1 July 2009 to 30 June 2011. The demographic information for each patient consists of the membership number, gender, birth date, and the visiting dates, where the frequency can be measured by counting the number of visits during the time period. Monetary value for each patient is excluded in this study since the majority of the income from a dental clinic is from the fixed copayment and registration fee per visit by national health insurance program in Taiwan [The descriptive statistics of length, recency, and frequency are summarized in  K-means method is performed for cluster analysis.  L, R, and F, average age and gender of the patients, and the symbol(s) of L, R, and F greater than the averages of L, R, and F. Most of the patients are in Clusters 3, 4, 8, 9, 10, and 12. Patients in Clusters 4 and 8 are younger children. More detailed analyses regarding gender and age group are reported in To perform cluster analysis, IBM SPSS Modeler 14.2 is the software, and the “Kohonen node” (viz. SOM) with default values is used. In addition, the Kohonen mode is set to “simple” for cluster analysis. The result generated by SOM suggests that the best number of clusters among 1,462 patients is twelve as shown in  L, R, and F values greater than the average L, R, and F values. Patients in Cluster 8 have larger R and F values. Clusters 1, 5, and 7 have higher L value compared with the average L value. Cluster 4 has larger R value, while Cluster 11 has larger F value. Finally, L, R, and F values in Clusters 3, 10, and 12 are smaller than the average L, R, and F values.From To further analyze patients' values, twelve clusters can be plotted by customer value matrix in terms of frequency and monetary. It is worth noting that monetary is assumed to be fixed in our study such that the original vertical axis (monetary) in  K-means method, along with LRFM model to cluster a total of 1,462 pediatric patients into twelve groups. From managerial viewpoints, patients in Cluster 11 belong to new customers. In addition to Cluster 11, patients in Cluster 8 are new customers with recent visits. The only difference between Cluster 11 and Cluster 8 is that patients in Cluster 8 have higher frequency to visit the dental clinic. In order to maintain a long-term relationship, the dental clinic needs to interact with the patients more often. In contrast to new customers, patients in Clusters 2, 6, and 9 belong to loyal customers. In order to increase the number of the visits, the dental clinic can provide better after-medical care activities. Finally, patients in Clusters 3, 10, and 12 with low length, recency, and frequency values might not be the focal point for marketing strategy since these patients contribute less to this dental clinic. More resources can be deployed elsewhere to create more profits for this clinic.This study uses a two-stage approach, namely, self-organizing maps and L, R, and F in From the viewpoints of customer value matrix, Clusters 2, 6, 8, 9, and 11 are the best customers. By directly observing the symbols of


In ball sports, we are taught to follow through, despite the inability of events after contact or release to influence the outcome . Here we •Future movements determine which motor memory is currently active and modifiable•Skills that otherwise interfere can be learned if each has a unique follow-through•A single skill is learned faster if its follow-through is consistent•Selection of motor memories depends on both lead-in and follow-through movements For a motor skill to be learned, its motor memory must be stored, protected from interference, and reactivated for modification during practice. Howard et al. demonstrate that future movement determines which motor memory is currently active and that a consistent follow-through constrains skill acquisition to a single memory, thereby speeding up learning. For a motor skill to be learned over a prolonged period of time, the motor memory of the skill must be stored, protected from interference by intervening tasks, and reactivated for modification when the skill is practiced. Given the widespread notion of the importance of a consistent follow-through in many sports , here we1,7 = 16.8; p = 0.005; hand paths shown in 1,7 = 17.706; p = 0.004) reaching around 40% of full compensation over a session. In contrast, when a second group of participants were presented with the final target, which again was predictive of the field direction, but did not follow-through to the target , suggesting that participants solely used non-specific co-contraction to reduce their error [1,124 = 7.388; p = 0.08) and force compensation , indicating that interference was strongly reduced in the follow-through group.Participants grasped the handle of a robotic interface  and madeir error . FinallyThese results show that . However, significantly faster learning was observed for the consistent, compared to the variable, follow-through group for both kinematic error  and force compensation  as shown using an ANOVA across all exposure trials with the main effects of experimental group and trial number. The same analysis, performed on the first one-third of the exposure trials, showed significantly faster learning for the consistent compared to the variable follow-through group for both kinematic error  and force compensation . By the end of the session, there were no significant differences in either the kinematic error  or force compensation  , showing that both groups eventually learned the same amount for this simple skill.We tested this prediction in two groups who experienced a single force field whose direction was fixed across all trials. Both groups made a movement in the force field to a central target followed by an unperturbed follow-through movement to a final target A. For onfast p < 0.001 between groups, other parameters non-significant), suggesting that variable follow-through leads to decreased retention across trials [We used a dual-rate model to examine whether changes in the parameters that govern learning might account for the differences that we observed in skill acquisition rate, but not final level of learning. The time course of learning novel dynamics is well accounted for by two interacting processes: a fast process that adapts and decays quickly and a slower process that adapts and decays more gradually . Each prs trials .Could our results on simple force-field learning over the course of an experimental session apply to real-world learning taking place over much more extended periods? In real-world tasks, such as a tennis or golf stroke, the lead-in to the movement is critical for task success, as it will determine characteristics such as variability at contact . Moreove1,5 = 36.750; p = 0.002) and increase in force compensation  to around 50%. Participants learned to access the motor memories based on the nonlinear rule as shown by the force compensation being of the appropriate sign for all four possible movements  have a very limited ability to separate motor memories , dynamicAlthough we have shown that consistent follow-through leads to faster learning through selection of a single memory, this does not preclude other potential advantages of the follow-though, such as injury reduction or other biomechanical advantages . AlthougI.S.H., D.M.W., and D.W.F. designed the study. I.S.H. ran the experiments. I.S.H., D.M.W., and D.W.F. performed the data analysis. I.S.H., D.M.W., and D.W.F. wrote the paper.


Banach  establisBanachX, d) be a complete metric space and f has a fixed point.Banach Let  be a complete metric space and T has a fixed point.Kannan Let b-quasi-metric family if Generating space of (2)b-dislocated metric family if Generating space of (3)b-dislocated-quasi metric family if Generating space of If y Banach .X, d) be a metric space. If we put d for all Let .Let open ball of radius x.Let closed ball of radius x.The set Now, we state some propositions and corollaries in In a n Kumari , Sarma an Kumari .LetthenIf we writeforthe operationsatisfies Kuratowski’s closure axioms of Kelley (b) be a metric space. Ifis a self mapping andx, ybe any elements ofX. Now consider the following contractive conditions:RhoadesThere exist nonnegative functionsa, b, csatisfyingRhoades There exsuch that, for eachCiric.1  be a complete metric space and ifis a self continuous mapping which satisfies:for allandThenThas a unique fixed point inX.We now give an example to illustrate the above corollary.Let d is a complete metric on X. Define the self mapping Clearly T.For Three eminent conditions (1), (2) and (3) are made significant contribution in the area of fixed point theory and applications. After these three results, a huge number of papers have been written by several authors to those results either improve or generalize some of the conditions (1), (2) or (3), or even the three conditions simultaneously.In 1972, Zamfirescu ((Zamfirescu1972) There exists real numbers Then T has a fixed point.In Rhoades , RhoadesRhoades 19a, b, c satisfying is a complete metric space. Ifbe a self continuous mapping satisfying any of the conditions either Rhodes or Zamfirescu or Ciric.1. ThenThas a unique fixed point.T has a unique fixed point. In Rhoades , Rhodes Letbe a complete-family with the parametersandIfis a self continuous mapping which satisfiess-h generating b-quasi-contraction, i.e.for allandIf for some positive integeris continuous, thenThas a unique fixed point inX.u in X.We can construct a sequence u. Also we have,Thus its subsequence u is a fixed point of Which yields that Now we shall prove that l be a smallest positive integer such that Let If Similarly,Thus Which is a contradiction.Hence u is a fixed point of T.Which yields X such that Now if there exists another point u is a unique fixed point of T in X. Hence If we take parameter Letbe a complete generating space of quasi-metric family andis a continuous mapping which satisfies:for allandIf for some positive integeris continuous, thenThas a unique fixed point inX.If we takethen we get following corollary.Let  be a complete metric space and ifis a continuous mapping which satisfies:for allandIf for some positive integeris continuous, thenThas a unique fixed point inX.Zamfirescu Contraction, we introduce the following definition in the setting of Based on the definition of Then f is called “s-z generating b-quasi-contraction”.Let Letbe a complete-family with the parametersIfbe a continuouss-z generating b-quasi-contractionthenfhas a unique fixed point inX.Put Now we prove that m, n are positive integers such that To do this,let Now considerThus Which implies there exist some f is continuous, we get Since u is a fixed point of f. Suppose there exists another point X such that Thus u is a unique fixed point.Since We now give an example to illustrate the above theorem. Let f satisfies the condition(1) of Theorem 34 but f doesn’t satisfy condition(2) of Theorem 34.Hence f in X.SinceIf we take parameter Letbe a complete generating spaces of quasi-metric family. Ifbe a continuous mapping such that for eachat least one of the following is true.Thenfhas a unique fixed point.If we takethen we get following corollary.Let  be a complete metric space. Ifbe a continuous mapping such that for eachat least one of the following is true.Thenfhas a unique fixed point.Open Question What are the additional conditions as needed in order to establish the existence of a unique fixed point satisfying the condition for any Conclusion In this work, we introduced a new concept of s-h generating b-quasi-contraction., s-z generating b-quasi-contraction.. Also, we derived the existence of fixed point theorems for generating spaces of b-quasi-metric family. Moreover, some examples are provided wherever necessary. Our results may be the motivation to other authors for extending and improving these results to be suitable tools for their applications.