To evaluate the center of pressure (COP) progression similarity and its change during walking and jogging in Anterior Cruciate Ligament deficient (ACLD) patients.A study was performed in 64 unilateral ACLD subjects and 32 healthy volunteers who walked and jogged on footscan® system at a self-selected speed. COP trajectory during walking and jogging was calculated. The robustness and similarity scores of COP  were computed, and then the Analysis of Variance test was employed to compare among different conditions .(1) During the same motion status , SSCOP were higher than 0.885. However, SSCOP between walking and jogging were lower than 0.25 in both the healthy and ACLD group. SSCOP between the intrasubjects were statistically higher than those between the intersubjects (p<0.01). (2) SSCOP in the ACLD group were statistically significantly reduced to 0.885±0.074 compared to 0.912±0.057 in healthy volunteers during walking, and 0.903±0.066 in the ACLD group compared to 0.919±0.050 in the healthy group during jogging (p<0.01).SSCOP can distinguish walking from jogging, and SSCOP of ACLD patients would be different from that of healthy controls.The study protocol was approved by the Institutional Research Board of Peking University Third Hospital (IRB00006761-2012010). Anterior cruciate ligament (ACL) deficiency is a common sports injury that increases tibiofemoral laxity and leads to knee joint instability, which can affect the performance of daily activities. Several biomechanical studies ,2,3,4 reGait cycle of the walking and running consists of a stance phase and a swing phase. The percentage of the stance phase varies depending on gait speed—about 60% with walking, 40% with running, and 22% with world class sprinters. The distinction of walking is that gait cycle involves a period of double limb support in which both feet are on the ground. However, there is a period of double float in which both feet are off the ground during jogging or running . The cenThe aim of this study was to describe and quantify the similarity of the COP displacement during walking and jogging in healthy volunteers and in unilateral ACL deficient patients. The similarity of the COP displacements during walking was further investigated whether differed from that during jogging, and if COP similarity in the unilateral ACL-deficient patients was different from that of healthy volunteers.Thirty-two healthy college students  and 64 symptomatically unilateral (33 left and 31 right side) ACL deficient subjects  participated in the study. The injured knee was verified with a physical examination by an orthopedic surgeon and with an MRI examination. Additionally, the status of each injured ACL was confirmed during an arthroscopy performed after the completion of this study. All ACL-deficient subjects included in this study had unilateral ACL rupture, accompanied with minimal other tissue injuries that do not need surgical intervention. There was no history of injury, surgery or disease in the contralateral knees. Regular intervention following ACL injury was done. Controlling the knee hemarthrosis and general inflammatory process was the first goal. The patients were given crutches and instructed in a pain-free partial weight-bearing gait. Motion exercises were started with concentration on passive extension to help prevent rapid scarring in the intercondylar notch. Weight bearing were increased as pain and joint effusion decreases, quadriceps control increases, and full active knee extension was achieved. The study protocol was approved by the Institutional Research Board of Peking University Third Hospital (IRB00006761-2012010). After explaining the aims and risks of the study, subjects gave written informed consent. During the test, all patients were free of knee joint swelling and were able to walk and jog without any subjective discomfort. However, they reported symptoms of giving-way during turning or twisting activities.The subjects walked or jogged barefoot on a two meter footscan® system , that was placed halfway on a 16.5 meter walkway. Before data collection, subjects warmed up and were asked to perform practice trials until they had familiarized themselves with barefoot walking and jogging. Subjects were asked to walk and jog at a constant (self chosen) speed.A trial was considered valid when the following criteria were met: (1) at least two complete footprints, (2) presence of a heel-strike pattern, (3) no visible adjustment in gait pattern when crossing the plate , (4) tot1) COP Trajectory GenerationCOPX and COPY, were the anterior-posterior, medio-lateral position of COP, respectively. Theoretically, they could be expressed according to following formula:F was the continuous pressure distribution in the supporting surface area. While in practice, pressure distribution was detected by pressure-sensor array, which just was a discrete case. To let the calculation of COPX and COPY be possible, formula  that satisfied the minimized process as characterized by the right side constraint of the formula. The vector  takes over all the activated sensors, and D, ) was the Euclidean distance between  and . The origin  of the coordinate system was set at the center of the heel.The coordinates of COP, noted as dxdywhere F Similarity ComputationS1 and S2 were two COP trajectories. Based on this definition, several statistics were defined to observe different aspects of human COP trajectory for a natural walking or jogging behavior. First, to observe the robustness of each person, an individual’s similarity referring to intra-participant similarity was defined as in formula  and Si, Sj were two intra-participant COP trajectories. To observe the characteristics of COP trajectories of a specific group of participants, the average among all individual similarities within the group was taken according to formula , between intrasubject (within the same subject) or intersubjects (between different subjects), and between walking or jogging.In order to evaluate whether the COP similarity under Walking motion pattern (marked in “W”) was different from that of Jogging motion pattern (marked in “J”) in Experiment I: On Different Motion Patterns, several groups corresponding to two motion patterns were setup under four conditions: Left side (marked in “L”), Right side (marked in “R”), Healthy volunteers (marked in “H”) and ACL-ruptured patients (marked in “A”). Thus, there was a total of four groups to investigate the difference between two motion patterns, noted as “HL-WJ”, “HR-WJ”, “AL-WJ” and “AR-WJ” respecitvely, where for example, the group labelled with “HL-WJ” referred that the observation were focused on Left side of Healthy subjects by calculating the similarity of COP trajectories between Walking and Jogging for a given subject. As a consequence, four similarity scores according to formula  were obtIn order to evaluate whether the COP similarity on the ipsilateral side (marked in “I”) was different from that on the contralateral sides (marked in “C”) in Experiment II: On ipsilateral and contralateral, similarity scores were calculated between the same sides and different sides  of the same subject during walking (marked in “W”) or jogging (marked in “J”). Several groups corresponding to two motion patterns were setup under four conditions: Left side (marked in “L”), Right side (marked in “R”), Healthy volunteers (marked in “H”), and ACL-ruptured patients (marked in “A”). Thus, there was a total of four groups were formed to investigate the difference between the contralateral sides, respectively noted as “HW-C”, “HJ-C”, “AW-C”, and “AJ-C”, where for example, the group labelled with “HJ-C” referred to the observation that focused on the Contralateral sides of Healthy subjects by calculating the similarity of COP trajectories during Jogging. As a consequence, four similarity scores according to formula  were obtIn order to evaluate whether the COP similarity on the sides of intrasubjects (marked in “Intra”) was different from that of the intersubjects (marked in “Inter”) in Experiment III: On intrasubject and intersubject, similarity scores were calculated between different sides of the same subject and those between different subjects during walking (marked in “W”) or jogging (marked in “J”). Several groups corresponding to two motion patterns were setup under four conditions: Left side (marked in “L”), Right side (marked in “R”), Healthy volunteers (marked in “H”), and ACL-ruptured patients (marked in “A”). Thus, there were a total of eight groups formed to investigate the difference between the sides of different subjects, respectively noted as “HWL-Inter”, “HWR-Inter”, “HJL-Inter”, “HJR-Inter”, “AWL-Inter”, “AWR-Inter”, “AJL-Inter” and “AJR-Inter”. For example, the group labelled with “HJR-Inter” referred to the observation focused on the Right side of the Healthy Intersubjects by calculating the similarity of COP trajectories during Jogging. As a consequence, eight similarity scores according to formula  were obtIn order to investigate whether the COP similarity changed following ACL rupture, differences between normal and pathology subjects were studied in Experiment IV: The influences derived from ACL deficiency. Firstly, similarity scores were calculated according to formula  for ACL-To be clearly, all results from above experiments were re-exhibited in a new way, where similarity score of a healthy volunteers group under given observation condition was taken as a control group, while that of a ACL deficiency group under the same condition was taken as a studied group. Each group under some observation condition was labeled by same characters just as that used in experiments mentioned above, only with some slight position adjustment to let it lay out more clearly. Moreover, for all observation conditions that involve both left and right foot, they were combined together by simple averaging. As a consequence, totally 11 studied group were formed, respectively noted as “A-W”, “A-J”, “A-WJ”, “A-W-I”, “A-W-C”, “A-J-I”, “A-J-C”, “A-W-Intra”, “A-W-Inter”, “A-J-Intra” and “A-J-Inter”, together with corresponding 11 control groups, noted as “H-W”, “H-J”, “H-WJ”, “H-W-I”, “H-W-C”, “H-J-I”, “H-J-C”, “H-W-Intra”, “H-W-Inter”, “H-J-Intra” and “H-J-Inter”. For example, the group labelled with “A-W” referred that the observation were focused on the ACL rupture patients during Walking, its similarity score was obtained by simply averaging the scores of both “AL-W” and “AR-W” in Experiment I; the group labelled with “A-J-C” was right the “AJ-C” in Experiment II, referred to the observation focused on the Contralateral side of ACL rupture patients while Jogging; the group labelled with “A-W-Intra” referred to the observation focused on the same ACL rupture subject during Walking, its similarity value was obtained by simply averaging the scores of both “AWL-Intra” and “AWR-Intra” in Experiment III; the group labelled with “A-J-Inter” referred to the observation focused on the intersubjects of ACL deficiency during Jogging, its similarity score was obtained by simply averaging the scores of both “JL-inter-A” and “JR-inter-A” in Experiment IV.To detect whether the difference among studied conditions was significant or not, the Analysis of Variance (ANOVA) test was employed. All statistical tests were performed using SPSS 15.0 . The level of statistical significance was set at 0.05.Left side of Healthy subjects during Walking. “HR-W” referred to that the observation focused on Right side of Healthy subjects during Walking. “HL-J” referred to that the observation focused on Left side of Healthy subjects during Jogging. “HR-J” referred to that the observation focused on Right side of Healthy subjects during Jogging. “HL-WJ” referred that the observation were focused on Left side of Healthy subjects by calculating the similarity of COP trajectories between Walking and Jogging. “HR-WJ” referred that the observation were focused on Right side of Healthy subjects by calculating the similarity of COP trajectories between Walking and Jogging.“HL-W” referred to that the observation focused on “HW-IL” referred to the observation focused on the Ipsilateral Left side of Healthy subjects during Walking. “HW-IR” referred to the observation focused on the Ipsilateral Right side of Healthy subjects during Walking. “HW-C” referred to the observation that focused on the Contralateral sides of Healthy subjects by calculating the similarity of COP trajectories during Walking. “HJ-IL” referred to the observation focused on the Ipsilateral Left side of Healthy subjects during Jogging. “HJ-IR” referred to the observation focused on the Ipsilateral Right side of Healthy subjects during Jogging. “HJ-C” referred to the observation that focused on the Contralateral sides of Healthy subjects by calculating the similarity of COP trajectories during Jogging.“HWL-Intra” referred to the observation focused on the Left side of the same Healthy subjects during Walking. “HWL-Inter” referred to the observation focused on the Left side of the Healthy Intersubjects by calculating the similarity of COP trajectories during Walking. “HWR-Intra” referred to the observation focused on the Right side of the same Healthy subjects during Walking.“HWR-Inter” referred to the observation focused on the Right side of the Healthy Intersubjects by calculating the similarity of COP trajectories during Walking.“HJL-Intra” referred to the observation focused on the Left side of the same Healthy subjects during Jogging. “HJL-Inter” referred to the observation focused on the Left side of the Healthy Intersubjects by calculating the similarity of COP trajectories during Jogging. “HJR-Intra” referred to the observation focused on the Right side of the same Healthy subjects during Jogging.“HJR-Inter” referred to the observation focused on the Right side of the Healthy Intersubjects by calculating the similarity of COP trajectories during Jogging.“WL-inter-H” referred to the observation focused on the Left side of the Healthy subjects by calculating the similarity of COP trajectories during Walking. “WL-inter-A” referred to the observation focused on the Left side of the ACL-ruptured patients by calculating the similarity of COP trajectories during Walking. “WL-HA” referred to the observation focused on the Left side between Healthy volunteers and ACL-ruptured patients by calculating the similarity scores of COP trajectories during Walking.“WR-inter-H” referred to the observation focused on the Right side of the Healthy subjects by calculating the similarity of COP trajectories during Walking. “WR-inter-A” referred to the observation focused on the Right side of the ACL-ruptured patients by calculating the similarity of COP trajectories during Walking. “WR-HA” referred to the observation focused on the Right side between Healthy volunteers and ACL-ruptured patients by calculating the similarity scores of COP trajectories during Walking.“JL-inter-H” referred to the observation focused on the Left side of the Healthy subjects by calculating the similarity of COP trajectories during Jogging. “JL-inter-A” referred to the observation focused on the Left side of the ACL-ruptured patients by calculating the similarity of COP trajectories during Jogging. “JL-HA” referred to the observation focused on the Left side between Healthy volunteers and ACL-ruptured patients by calculating the similarity scores of COP trajectories during Jogging. “JR-inter-H” referred to the observation focused on the Right side of the Healthy subjects by calculating the similarity of COP trajectories during Jogging. “JR-inter-A” referred to the observation focused on the Right side of the ACL-ruptured patients by calculating the similarity of COP trajectories during Jogging. “JR-HA” referred to the observation focused on the Right side between Healthy volunteers and ACL-ruptured patients by calculating the similarity scores of COP trajectories during Jogging.From the re-exhibition of these results in , it can “H-W” referred that the observation were focused on the Healthy subjects during Walking, its similarity score was obtained by simply averaging the scores of both “HL-W” and “HR-W” in Experiment I; “H-J-C” referred to the observation focused on the Contralateral side of the Healthy subjects while Jogging; “H-W-Intra” referred to the observation focused on the same the Healthy subjects during Walking; “H-J-Inter” referred to the observation focused on the intersubjects of the Healthy subjects during Jogging.“A-W” referred that the observation were focused on the ACL rupture patients during Walking, its similarity score was obtained by simply averaging the scores of both “AL-W” and “AR-W” in Experiment I; the group labelled with “A-J-C” is right the “AJ-C” in Experiment II, referred to the observation focused on the Contralateral side of ACL rupture patients while Jogging; the group labelled with “A-W-Intra” referred to the observation focused on the same ACL rupture subject during Walking, its similarity value was obtained by simply averaging the scores of both “AWL-Intra” and “AWR-Intra” in Experiment III; the group labelled with “A-J-Inter” referred to the observation focused on the intersubjects of ACL deficiency during Jogging, its similarity score was obtained by simply averaging the scores of both “JL-inter-A” and “JR-inter-A” in Experiment IV.The purpose of this study was to examine the similarity of the COP and its change during walking and jogging following the ACL deficiency. In accordance with our hypotheses, COP similarity can distinguish walking from jogging. In addition, the COP similarity would be changed in ACL-ruptured patients compared with healthy volunteers. These findings provide a reliable dataset for COP characteristics during walking and jogging in healthy volunteers and ACL-ruptured patients.Walking and jogging are the most common activities that humans perform during activity of daily living, sports and rehabilitation. Given that an average adult takes more than 5,000 steps per day , even slSimilarity scores between walking and jogging COP trajectories were lower than 0.25 see  which inAlthough similarity scores between the COP trajectories in healthy volunteers were high , there was a significant difference between left and right sides not only during walking but also during jogging (p<0.0001). Our results showed that asymmetry and laterality (limb preference for a particular task) previously recognized not only in temporal and force parameters ,22,23 buACL deficiency would result in knee joint instability and increase internal rotation of the tibia during the gait cycle, which wIn summary, the present study showed that COP similarity from the dynamic pedobarography not only provides a practical and objective way to evaluate dynamic function of walking and jogging that was not studied exclusively before, but also helps in evaluating and guiding future rehabilitative and surgical techniques in ACL deficient patients. However, previous studies have established that ACL rupture induces variability in postural control and gait patterns. ACL deficiency adopts certain strategies to maintain dynamic stability and balance. The modulating mechanism for postural control is complex, possibly affecting the COP trajectory. Interpretations of these findings should not only generalize the effect of ACL rupture, but also consider the variation in dynamic postural control during motion. Further studies are needed to clarify the relationship between dynamic stability and ACL rupture. Furthermore, large-scale measurements of muscle activity and more detailed kinematic studies in both ACL-intact and deficient subjects are necessary to develop population reference values to improve our general clinical understanding of individual variations in the biomechanical coupling between foot and knee.This investigation showed the COP similarity during walking was significantly different from that during jogging. ACL deficiency tends to display a more variable COP curve compared to healthy adults. These findings regarding the COP similarity for ACL rupture provided more information for dynamic foot function and gait performance, potentially assisting in clinical intervention assessment for the ACL rupture.S1 Dataset(ZIP)Click here for additional data file.


Cell-penetrating peptides (CPPs) are short peptides (5–30 amino acids) that can enter almost any cell without significant damage. On account of their high delivery efficiency, CPPs are promising candidates for gene therapy and cancer treatment. Accordingly, techniques that correctly predict CPPs are anticipated to accelerate CPP applications in future therapeutics. Recently, computational methods have been reportedly successful in predicting CPPs. Unfortunately, the predictive performance of existing methods is not satisfactory and reliable so as to accurately identify CPPs.In this study, we propose a novel computational predictor called SkipCPP-Pred to further improve the predictive performance. The novelty of the proposed predictor is that we present a sequence-based feature representation algorithm called adaptive k-skip-n-gram that sufficiently captures the intrinsic correlation information of residues. By fusing the proposed adaptive skip features with a random forest (RF) classifier, we successfully construct the prediction model of SkipCPP-Pred. The various jackknife results demonstrate that the proposed SkipCPP-Pred is 3.6% higher than state-of-the-art CPP predictors in terms of accuracy. Moreover, we construct a high-quality benchmark dataset by reducing the data redundancy and enhancing the similarity between the positive and negative classes. Using this dataset to build prediction models, we can successfully avoid the performance bias lying in existing methods and yield a promising predictive model.http://server.malab.cn/SkipCPP-Pred/Index.html).The proposed SkipCPP-Pred is a simple and fast sequence-based predictor featured with the adaptive k-skip-n-gram model for the improved prediction of CPPs. Currently, SkipCPP-Pred is publicly available from an online webserver (The online version of this article (10.1186/s12864-017-4128-1) contains supplementary material, which is available to authorized users. Cell-penetrating peptides (CPPs) are short peptides usually comprising 5–30 amino acid residues. Also known as protein transduction domains (PTDs), membrane translocating sequences (MTSs), and Trojan peptides, CPPs can directly enter cells without significantly damaging the cell membrane –3. This The first CPP, namely the Tat peptide, was derived from the transcription activator of human immunodeficiency virus type 1 in the late 1980s . Since tTwo factors , featureAs mentioned above, it is argued that the high predictive performance of existing computational methods seems doubtful –21. Firshttp://server.malab.cn/SkipCPP-Pred/Index.html), and is anticipated to become an efficient tool for researchers working with CPPs.In this study, we propose a high-quality dataset for predicting CPPs. The high-quality of the dataset lies in three aspects. First, the dataset shares relatively low sequence similarity, with no more than 80%, avoid the bias in the performance. Second, the new CPP dataset is sufficiently large to build prediction models. Third, the dataset considers the importance of negative samples on predictive performance . The colFigure The k-skip-n-gram model, pioneered by Guthrie et al. , integraS as A1A2A3…AL-1AL, where L represents the length of the sequence; and the indices denote the positions of the amino acids in the sequence . Ai (1 ≤ i ≤ L) belongs to a set of 20 different amino acids, alphabetically ordered as Ω = {A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, V, W, Y}. An element of Ω is denoted as i′ ≤ 20; for example, Ω1 and Ω2 denote the first and seconds elements of Ω, respectively.For convenience, we denote a given amino acid sequence AiAi + 1…Ai + n-1) in the sequence S. To transform the variable sequence length into fixed length feature vectors, the traditional n-gram features are computed as follows:The traditional n-gram model provides the composition of n contiguous residues  denotes the total number of all elements in the set Tgrams. n possible residue combinations with length n. Tgrams. Accordingly, FVGrams measures the occurrence frequencies of S. The dimension of FVGrams is 20n.where S is short. To address this problem, the k-skip-n-gram model integrates the distance information into the n-gram model. The distance between any two residues Ai and Aj in a given sequence S is given by the interval length between the residues, calculated asClearly, the traditional n-gram model is sparse when the sequence A1 and A2 are contiguous, they are separated by an interval of zero length , and DT = 0, A1 and A3 are separated by one residue (A2), so the interval length DT = 1. Similarly, DT = 2 indicates that A1 and A4 are separated by two residues (A2 and A3).For example, if n residues with distances ≤k in S. In other words, in addition to the n contiguous residues considered in the traditional n-gram model, this model considers the n residues with distances 1 to k in S. Similar to the n-gram features, the k-skip-n-gram features are calculated asThe k-skip-n-gram model provides the composition of N(TSkipGram) denotes the total number of all elements in the set TSkipGram, and TSkipGram, which is formulated aswhere Skip(DT = a) = {AiAi + a + 1 … Ai + a + n − 1| 1 ≤ i ≤ L − a ,  1 ≤ a ≤ k}.where n, leading to the overfitting problem, we limit our analysis to n<3 (<203-D). Moreover, when n = 1, the k-skip-n-gram model reduces to the traditional n-gram model. Thus, we analyze only the case for n = 2. The feature extraction procedure of the k-skip-2-g model is depicted in Fig. Because the dimensions of the feature space exponentially expand with  to n<3 <03-D. Mork measures the distance between any two residues. The maximum value of k should be the minimum sequence length in the dataset. However, almost all CPP sequences are short (5–30 residues), implying that k should not exceed 5. In other words, the skip features consider only the local distance information with the interval length no more than 5 in all sequences in the dataset, which would not properly reflect the varying distances in the dataset. Therefore, we proposed a modified strategy in which k is the length of each sequence during the skip feature extraction. In this way, the proposed feature representation algorithm becomes parameter free. Additionally, the feature algorithm adapts to the different lengths of sequences in the dataset, and includes more distance information in the features. The k-skip-n-gram features extracted by this new strategy are referred to as adaptive k-skip-n-gram features.By definition of the k-skip-n-gram model, the parameter The RF classifier, introduced by Breiman et al. , has proAccording to machine learning theory, a well-established dataset is essential for building a robust and reliable prediction model. In this study, we carefully constructed an updated high-quality stringent dataset. The construction process of our dataset is described below.A CPP prediction dataset includes positive members  and negative members (non-CPPs). The initial positive dataset was constructed from 1855 experimentally validated CPPs downloaded from the CPPsite2.0 database. Among these are 1564 natural and 291 non-natural CPPs. Natural CPPs are natural amino acid sequences, whereas in non-natural CPPs, some of the amino acids are replaced by artificial characters. As these non-natural CPPs cannot be formulated into fixed length feature vectors, they are excluded from the positive dataset. Redundancy in the dataset is known to bias the predictive performance of a model. Therefore, to improve the quality of the positive dataset, we removed the redundancies using the CD-HIT program , which hTo construct the negative dataset, we randomly generated 2452 amino acid sequences with lengths between 5 and 50 as our initial negative pool. Such random generation of negative samples has been adopted in previous studies , 11. ForUltimately, we successfully constructed a high-quality dataset containing 462 CPPs (positive samples) and 462 non-CPPs (negative samples). For convenience, our dataset is designated as CPP924. To our knowledge, CPP924 has the lowest data redundancy and the highest between-class similarity among the reported datasets. By virtue of the latter quality, our dataset is highly representative of real CPPs.The quality of the predictor was evaluated by evaluation metrics and a validation method.Here, we employed four metrics commonly used in binary predictor evaluations ; sensitiTP, TN, FP and FN represent the numbers of true positives, true negatives, false positives and false negatives, respectively. The first two metrics, SE and SP, measure the ability of the predictor to predict the correct class [where ct class . Specifik-fold cross validation, the jackknife test [The effectiveness of a prediction model must be tested by a validation method. The three statistical validation methods are ife test , and an ife test –54. The Step 1. Go to the website  to see the homepage of the webserver. Click on the button About and you will see a brief introduction about how the proposed method SkipCPP-Pred is set up.Step 2. Enter the query protein sequences into the input box. The input sequences should be in the FASTA format. Examples of FASTA-formatted protein sequences can be seen by clicking on the button FASTA format above the input box. In particular, the webserver can receive an un-limited number of query sequences for every single run.Step 3. By clicking on the button Predict, you will get the predicted results on the screen of your computer. Take an actual cell-penetrating peptide, with an identifier of “cpp_P7–4”, as an example. After you enter the query sequence into the input box and click on the button Predict, you will see the predicted result showed on the screen: “Cell-penetrating” with the prediction confidence of 98.8%.Step 4. Click on the button Clear to delete the query sequences you enter to the input box.Step 5. Click on the button Datasets to download the benchmark datasets used in this paper and the feature sets we proposed based on the datasets.An available webserver is important for researchers to access the proposed method to make predictions. Here, we have built a user-friendly webserver that implements the proposed method SkipCPP-Pred. The webserver is now freely accessible to the public. In this section, we give researchers a step-by-step guideline on how to use the webserver to get the predicted results they desire. The guideline is described as follows,This study proposes adaptive k-skip-2-g features as modifications of the traditional 2-g features. To investigate the impact of the adaptive k-skip-2-g features, we compared their performances with those of the traditional 2-g features. Classifier bias was avoided by employing two high-efficiency classifiers (RF and LibSVM). Table We also analyzed the importance of the proposed k-skip-2-g features by calculating the information gain score . This met, which can be any integer higher than 1. Here, we investigated the impact of varying t from 10 to 500 in 10-step increments, and executing the RF classifier on the CPP924 dataset. The jackknife results for the various t values are illustrated in Fig. t = 150. Therefore, we set t = 150 in our prediction model. The prediction results of the RF classifier for different values of t are detailed in Additional file Classifier parameter optimization is a potentially useful means of improving the predictive performance of machine learning methods –60. TherTable To evaluate the effectiveness of the proposed computational predictor SkipCPP-Pred, we compared its performance with that of CellPPD, the best-performing predictor in the literature . In thisThe prediction results of the proposed SkipCPP-Pred and CellPPD, executed on the proposed high-quality dataset and evaluated by the jackknife test, are presented in Table To intuitively compare the proposed method with the CellPPD predictor, we further conducted a graphic analysis by using the Receiver Operating Characteristic (ROC) curves . In the To identify known and potential novel CPPs, we developed a predictor, namely SkipCPP-Pred, by using a sequence-based feature representation approach (adaptive k-skip-n-gram model).To verify the effectiveness of the adaptive k-skip-n-gram model, we compared it with traditional n-gram model and found that the adaptive k-skip-n-gram model shows better performance than the traditional n-gram model. This is because that as compared with the tradition n-gram model, additional distanced correlation information embedded in the adaptive k-skip-n-gram model contributes to the performance improvement for CPP prediction.Moreover, due to the use of only sequential information for modeling, SkipCPP-Pred is capable to predict CPPs fast. We tested SkipCPP-Pred on the stringent CPP924 dataset with the jackknife test. The results indicated that SkipCPP-Pred has the strong capacity of classifying true CPPs from non-CPPs. And, we developed an online webserver that implements the proposed SkipCPP-Pred for researchers to predict CPPs conveniently. It is anticipated to be a useful tool to accelerate the research of CPP prediction. Importantly, the proposed feature representation method used in SkipCPP-Pred has the great potential to guide the sequence-based prediction of other special proteins (i.e. DNA-binding proteins).In this study, we proposed a novel computational method called SkipCPP-Pred, for accurate, fast and stable prediction of potential novel CPPs. Recognizing the importance of the dataset in model building, we also proposed a novel high-quality dataset for SkipCPP-Pred. The quality of this dataset is guaranteed by reducing the sequence redundancy, which alleviates the bias in the performance, and enhancing the similarity between the two classes (positive and negative CPPs). To our knowledge, we present the most stringent of the datasets reported in the literature. Thus, our high-quality dataset might become the benchmark dataset in the development of computational CPP prediction methods. As another contribution, we proposed the adaptive k-skip-n-gram model to CPP prediction. By the feature comparative analysis, the k-skip-n-gram feature model demonstrated greater discriminative power in CPP classification than the traditional n-gram model. Moreover, we compared the overall performance of the proposed SkipCPP-Pred and the state-of-the-art predictors in the literature. The jackknife results showed that the ACC and MCC performance measures on the CPP924 dataset were higher in SkipCPP-Pred than in the state-of-the-art predictors, demonstrating the superiority of SkipCPP-Pred. Accordingly, it is expected that the proposed predictor could become a useful tool in research of CPP prediction. At least, it could complement the existing predictors to improve the accuracy of CPP prediction by neural-like computing models –67, evolAdditional file 1: Table S1.a denotes information gain score. Higher IG for a feature means the feature is more discriminative. Table S2. Performance of the Random Forest classifier with different tree numbers on the benchmark dataset CPP924 with the jackknife validation test. Note that the tree number is changed from 10 to 500 with the incremental step of 10. (DOCX 61 kb)Feature ranking of the proposed adaptive k-skip-2-g features. IG


S-type eigenvalue localization set for a tensor is derived by dividing S and its complement. It is proved that this new set is sharper than those presented by Qi (J. Symb. Comput. 40:1302-1324, et al. (Numer. Linear Algebra Appl. 21:39-50, et al. (Linear Algebra Appl. 481:36-53, H-eigenvalue of strong M-tensors are established, and we prove that these bounds are tighter than those obtained by Li et al. (Numer. Linear Algebra Appl. 21:39-50, In this paper, a new  Eigenvalue problems of higher order tensors have become an important topic in the applied mathematics branch of numerical multilinear algebra, and they have a wide range of practical applications, such as best-rank one approximation in data analysis , higher H-eigenvalue of a tensor, consequently, one can identify the positive (semi-)definiteness of the multivariate homogeneous polynomial determined by this tensor; for details, see [One of many practical applications of eigenvalues of tensors is that one can identify the positive (semi-)definiteness for an even-order real symmetric tensor by using the smallest ils, see , 21, 22.H-eigenvalue of tensors when the order and dimension are very large, we always try to give a set including all eigenvalues in the complex. Some sets including all eigenvalues of tensors have been presented by some researchers [H-eigenvalue is positive, consequently, the corresponding tensor is positive definite. Therefore, the main aim of this paper is to study the new eigenvalue inclusion set for tensors called the new S-type eigenvalue inclusion set, which is sharper than some existing ones.However, as mentioned in , 23, 24,earchers –3, 21–24n, N denotes the set m dimension n, denoted by For a positive integer =1,2,…,m .H-eigenpair, if both λ and x are real [Let cument}A  if \docu,xnm−1)T . Furtherm dimension n is called the unit tensor [A real tensor of order t tensor , denotedm-order n-dimensional tensor Z-tensor, if all of its off-diagonal entries are non-positive, which is equivalent to writing m-order and n-dimensional Z-tensors. A Z-tensor M-tensor if M-tensor if An negative , 14, 26,t}s>ρ(B) , 27.M-tensor m indices.The tensor educible . The speent}ρ(A)  of the tcument}A . A real mentclass2pt{minimymmetric , 22, 23 Let H-eigenvalue of nonsingular M-tensors. Wang and Wei [H-eigenvalue of nonsingular M-tensors, and they showed those are better than the ones in [S-type eigenvalue inclusion set, the other main results of this paper is to provide sharper bounds for the spectral radius of nonnegative tensors and the minimum H-eigenvalue of nonsingular M-tensors, which improve some existing ones.Recently, much literature has focused on the bounds of the spectral radius of nonnegative tensor in , 24, 28. and Wei  presenteBefore presenting our results, we review the existing results that relate to the eigenvalue inclusion sets for tensors. In 2005, Qi  generaliLetThenwhereis the set of all the eigenvalues ofandet al. [To get sharper eigenvalue inclusion sets than entclass1pt{minimai et al. , 30 and LetThenwhereet al. [S-type eigenvalue localization set by breaking N into disjoint subsets S and S̄, where S̄ is the complement of S in N.In addition, in order to reduce computations of determining the sets entclass1pt{minimaLetandSbe a nonempty proper subset of N. Thenwhereoris defined as in Lemma et al. [Based on the results of , in the et al.  exhibiteLetandSbe a nonempty proper subset of N. ThenwhereS-type eigenvalue inclusion set for tensors. It is proved to be tighter than the tensor Geršgorin eigenvalue inclusion set S-type eigenvalue localization set H-eigenvalue of strong M-tensors. Numerical examples are implemented to illustrate this fact.In this paper, we continue this research on the eigenvalue inclusion sets for tensors; inspired by the ideas of , 3, we oS-type eigenvalue inclusion set for tensors is given, and proved to be tighter than the existing ones derived in Lemmas H-eigenvalue of strong M-tensors, which are proved to be sharper than the ones obtained by He and Huang [The remainder of this paper is organized as follows. In Section nd Huang . FinallyIn this section, we start with some lemmas on tensors. They will be useful in the following proofs.Ifis irreducible nonnegative, thenis a positive eigenvalue with an entrywise positive eigenvectorx, i.e., corresponding to it.Letbe a nonnegative tensor. ThenSuppose thatThenLetbe a strongM-tensor and denoted bythe minimum value of the real part of all eigenvalues ofThenis an eigenvalue ofwith a nonnegative eigenvector. Moreover, ifis irreducible, thenis a unique eigenvalue with a positive eigenvector.Letbe an irreducible strongM-tensor. ThenA tensoris semi-positive if and only if there existssuch thatAZ-tensor is a nonsingularM-tensor if and only if it is semi-positive.Letassume thatis anM-tensor andThenis anM-tensor, andS-type eigenvalue set for tensors and establish the comparisons between this new set with those in Lemmas In this section, we propose a new LetwithAnd letSbe a nonempty proper subset ofN. Thenwhereλ, i.e., For any i.e., (i) entclass1pt{minima(ii) entclass1pt{minimaS. If S to make Note that S-type eigenvalue inclusion set derived in this paper and those in Lemmas Next, we establish a comparison theorem for the new LetwithThenz satisfies According to Remark entclass1pt{minimaBased on the results of Section Letbe an irreducible nonnegative tensor withAnd letSbe a nonempty proper subset ofN. ThenwherewithSince i.e., (i) mentclasspt{minimai.e., (ii) Next, we extend the results of Theorem Letbe a nonnegative tensor withAnd letSbe a nonempty proper subset ofN. Thenwherewithε denotes the tensor with every entry being 1. Then i.e., m and n are finite numbers, then by the properties of the sequence, it is easy to see that Let m 2.3 in , we haveed bound . □Now, we compare the upper bound in Theorem entclass1pt{minimaWe have showed that our bound is sharper than the existing one in . Now we Let entclass1pt{minimaentclass1pt{minimaH-eigenvalue of strong M-tensors and derive sharper bounds for that. This bounds are proved to be tighter than those in Theorem 2.2 of [In this section, by making use of the results in Section m 2.2 of .Letbe an irreducible nonsingularM-tensor withAnd letSbe a nonempty proper subset ofN. ThenwhereandwithM-tensor, by Lemma Since i.e., (i) ows from  that \do(ii) ows from  that \doWe next prove the bounds in Theorem M-tensor: Consider the following irreducible nonsingular m 2.1 of , we havem 4.5 of , we get In the sequel, we extend the results of Theorem Letbe a nonsingularM-tensor withAnd letSbe a nonempty proper subset ofN. ThenwhereandwithM-tensor, k is a positive integer, we see that the Z-tensor M-tensor if M-tensors satisfying i.e., λ is an eigenvalue of M-tensor for Since S-type eigenvalue inclusion set for tensors is presented, which is proved to be sharper than the ones in [H-eigenvalue of strong M-tensors, these bounds improve some existing ones obtained by Li et al. [In this paper, a new  ones in , 3. As ai et al.  and He ai et al. . In addiS-type eigenvalue inclusion set and the derived bounds depend on the set S. How to choose S to make S.However, the new


The Need for Bayesian Hypothesis Testing in Psychological Scienceby Wagenmakers, E. J., Verhagen, J., Ly, A., Matzke, D., Steingroever, H., Rouder, J. N., et al. (2017). Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions, eds S. O. Lilienfeld and I. D. Waldman (Chichester: JohnWiley & Sons), 123–138.A commentary on Psychological Science under Scrutiny. Their primary goal was to demonstrate the illogical nature of p-values, while, secondarily, they would also defend the philosophical consistency of the Bayesian alternative. In my opinion, they achieved their secondary goal but failed their primary one, thereby this contribution. I will, thus, comment on their interpretation of the logic underlying p-values without necessarily invalidating their Bayesian arguments.Wagenmakers et al.  argued tp-value itself, the statistic used by frequentists when testing research data.Historical criticisms . Therefore, their first premise proposes that, if H0 is true, the observed p-values cannot occur ; therefore not H0      (1)If Hmodus tollens is rendered inconsistent when made probabilistic, as follows:Wagenmakers et al.'s If H0—is unlikely. Therefore, the syllogism could be simplified as follows:There are, however, three problems with (2), problems which I would like to comment upon. One problem is stylistic: The first premise states a redundant probability; that is, that a significant result—which already implies an unlikely or improbable event under H0, then p < sig; p < sig; therefore probably not H0      (3)If Hmodus tollens  now highlights another of the problems: The second premise simply affirms that an unlikely result just happened If HModus tollens is in the form, If P, then Q; not Q; therefore not P. Thus, whenever the consequent (Q) gets denied in the second premise, it leads to denying the antecedent (P) in the conclusion. Such operation ought to prevail with probabilistic premises, as well  may be denied without its probability warranting transposition onto a non-probabilistic antecedent (P). For example, if all red cars (P) have a 95% chance of getting stolen (Q ≥ 0.95) and we learn of a Lamborghini with a lesser probability of so disappearing (not Q ≥ 0.95), it is logical to conclude that the Lamborghini is not red (not P).Correction (4) brings to light the most important problem: P > 0.2), yet he is not submitting his latest article (not Q), it is logical to conclude that he probably expects no publication (not P > 0.2).In comparison, Bayesian logic allows for the antecedent to be probable. For example, if John always submits to Nature (Q) whenever his subjective probability of getting published soars above 20% (modus tollens (1). Said otherwise, while Bayesian statistics allow for the antecedent to be probable (Pp), Fisher's and Neyman-Pearson's tests assume exact antecedents (P); therefore, a probabilistic conclusion does not hold with frequentist tests ] (Perezgonzalez, reductio ad absurdum argument may be informed by, but it is not dependent on, such p-value, the reductio being determined exclusively by the chosen level of significance. For “it is open to the experimenter to be more or less exacting in respect of the smallness of the probability he would require before he would be willing to admit that his observations have demonstrated a positive result. It is obvious that an experiment would be useless of which no possible result would satisfy him” (Fisher, It ought to be noted that the  Fisher, , p.13.modus tollens logically. Wagenmakers et al.'s criticism of the p-value is faulty in that they allow for a probability transposition not warranted either by modus tollens or by the technical apparatus of Fisher's and of Neyman-Pearson's tests. This critique, however, does not extend to their Bayesian argumentation, an approach much needed for testing hypotheses—rather than just testing data—in contemporary science.In conclusion, the technology of frequentist testing holds their The author confirms being the sole contributor of this work and approved it for publication.The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Using the existence of solutions for equilibrium equations with a Neumann type boundary condition as developed by Shi and Liao (J. Inequal. Appl. 2015:363,  L on the space T, there exists a unique Borel regular measure μ on T such that T; as an application, Shi and Liao , by Rad}Rn (see ), by Ban}Rn . He. HeL on aps .E be a Banach space and E and X and Y are subsets of E we shall denote by X is denoted by X is the function E. We endowed δ and the structures T be a non-empty set, let T and let T, endowed with the topology of uniform convergence. We denote by i.e. of the form A and σ-algebra, then M is additive if A, B in M is said to be bounded if E. The semivariation of M is the map A defined byA; M will be called a set-valued measure of finite semivariation. We denote by m be a vector measure from E. We say that m is a bounded additive vector measure if its verifies similar conditions of bounded additive set-valued measures. We denote by m defined by A; L is an additive (resp. positively homogeneous) if for all f defined on T, we set Let ace  oLetbe the setand letbe a continuous linear map. Thenlis associated with an additive, positively homogeneous and continuous set-valued map if and only iffor allj is an isomorphism, more a homeomorphism \documeLetLbe an additive, positively homogeneous and continuous set-valued map fromtoThen there is a unique bounded additive set-valued measureMfromtosuch thatfor allConversely for all bounded additive set-valued measurethe map: fromtois an additive, positively homogeneous and continuous set-valued map.M. Let l the unique extension of M from The second part follows from the definition of the integral with respect to The following corollary is partly known .


The LLOQ for AB-PINACA, AB-PINACA OH, and AB-PINACA pent is 0.10 ng/mL; the LLOQ for mitragynine is 1.0 ng/mL.” The online version is correct.In the report “Synthetic Cannabinoid and Mitragynine Exposure of Law Enforcement Agents During the Raid of an Illegal Laboratory — Nevada, 2014,” on page 1292, the following footnote should have been included with the Table: “


This “naive” approach is, however, suboptimal in terms of statistical power, as it ignores information about the intra-subject variance. To address this issue, we review several approaches to deal with nested data, with a focus on methods that are easy to implement. With what we call the sufficient-summary-statistic approach, we highlight a computationally efficient technique that can improve statistical power by taking into account within-subject variances, and we provide step-by-step instructions on how to apply this approach to a number of frequently-used measures of effect size. The properties of the reviewed approaches and the potential benefits over a group-level t-test are quantitatively assessed on simulated data and demonstrated on EEG data from a simulated-driving experiment.Hierarchically-organized data arise naturally in many psychology and neuroscience studies. As the standard assumption of independent and identically distributed samples does not hold for such data, two important problems are to accurately estimate group-level effect sizes, and to obtain powerful statistical tests against group-level null hypotheses. A common approach is to summarize subject-level data by a single quantity per subject, which is often the mean or the difference between class means, and treat these as samples in a group-level  In psychology and neuroimaging, for example, multiple data points are often acquired for the same subject throughout the course of an experiment; thus, there exists a subject (lower) and a group (higher) level in the data hierarchy. Two important questions are how to obtain precise estimators for group-level effect sizes from nested data, and how to obtain powerful statistical tests for the presence of group-level effects. The main difficulty associated with such nested data is that the assumption of identically distributed observations is typically violated: while samples acquired from the same subject can be considered to be identically distributed, different distributions must be assumed for different subjects. Therefore, simply pooling the data of all subjects in order to apply a standard statistical test like a A flexible way to model nested data is to combine the data of all subjects in a single linear model, referred to as the nested linear model, hierarchical linear model, multi-level model or linear mixed model  literature, the use of suboptimal inference procedures is currently still widespread, as discussed in Mumford and Nichols  and Pernsufficient summary statistics of separate subject-level analyses. The resulting statistical methods are simple to implement, computationally efficient, and can be easily extended to settings with more than two nesting levels, which are common, e.g., in the analysis of functional magnetic resonance imaging (fMRI) data.Group-level statistical power can be improved by incorporating variances at the lower level in relatively simple ways. The problem of estimating group-level effect sizes and estimating their statistical significance can, moreover, be formulated in a compellingly simple framework, where group-level inference is conducted using the Sufficient summary statistics approaches are popular in the field of meta analysis  in relation to summary-statistic approaches.With this paper, we aim to fill this gap by providing a review of ways to estimate group-level effect sizes and to assess their statistical significance in the context of neuroimaging experiments. We first provide a reference for a number of popular parametric and non-parametric effect size measures (section 2.2), which may be skipped by readers who want to proceed directly to the nested setting. We then discuss the need to choose an appropriate group-level model, as between-subject variability differs depending on whether a “random effects” or “fixed effect” model is assumed (section 2.3). We also demonstrate why the simple approach of ignoring the group structure by pooling the data of all subject is invalid (section 2.4). We then outline the popular “naive” summary-statistic approach of computing effect sizes on the subject level and treating these effect sizes as single samples in a group-level test (section 2.5). With what we call the sufficient-summary-statistic approach, we then discuss a family of techniques capable to yield unbiased group-level effect size estimates and powerful statistical tests of group-level null hypotheses, and we highlight a particular approach that yields minimum-variance effect size estimates by weighting each effect with the inverse of its variance. In a tutorial style, we outline the steps that are required to apply this approach to different effect size measures (section 2.6). Lastly, we discuss the advantages and drawbacks of Stouffer's method of combining subject-level Using synthetic data representing a two-sample separation problem, we empirically assess the performance of the reviewed approaches (section 3). The properties of the various approaches and the advantages of the sufficient-summary-statistic approach are further highlighted in an application to EEG data acquired during simulated emergency braking in a driving simulator (section 4). All data are provided in Matlab format along with corresponding analysis codeThe paper ends with a discussion of nested linear models, of multivariate extensions, and a note on non-parametric (bootstrapping and surrogate data) approaches (section 5).effect size θ is any quantitative measure that reflects the magnitude of some phenomenon of interest . An estimatorunbiased, if its expected value is θ.An statistical test is a procedure to decide, based on observed data, whether a hypothesis about a population is true. In this paper, our goal is to make inference about the presence or absence of an effect in the population. The null hypothesis is that no effect is present. The zero effect is denoted by θ0. The null hypothesis of no effect is denoted by H0 : θ = θ0. The alternative hypothesis that an effect is present is denoted by H1. A one-tailed alternative hypothesis assumes that either H1 : θ > θ0 or H1 : θ < θ0, while a two-tailed alternative hypothesis assumes that H1 : θ ≠ θ0.A test statistic needs to be derived from the observed effect size, where its distribution under the null hypothesis is known or can be reasonably well approximated. The p-value denotes the probability of obtaining a result at least as strong as the observed test statistic under the assumption of the null hypothesis. Denoting the test statistic by T, its cumulative distribution function under the H0 by FT, and its observed value in a given sample by τ, the p-values for a one-tailed alternative hypothesis are given byA Pr(·) denotes probability. The p-value for a two-tailed alternative hypothesis is given bywhere p-value falls below an alpha-level α. In the opposite case, no conclusion is drawn. The most commonly used alpha-levels are α = 0.05 and α = 0.01. If the null hypothesis is rejected, we speak of a statistically significant effect. The value of the test statistic that is required for a significant effect is called critical value. The power of a statistical test is the probability that the test correctly rejects the null hypothesis when the alternative hypothesis is true. Conversely, the false positive rate is the fraction of non-existent effects that are statistically significant.The null hypothesis is rejected if the Before introducing the nested data setting, we review a number of popular effect size measures. For each measure, we also present an analytic expression of its variance, which is a prerequisite for assessing its statistical significance. We will later need the variance for performing statistical inference in the nested setting, too.N independent samples x1, …, xN ∈ ℝ. The sample mean is denoted by A common measure of effect size is the mean of a sample. Consider a neuroimaging experiment, in which the participant is repeatedly exposed to the same stimulus. A common question to ask is whether this stimulus evokes a brain response that is significantly different from a baseline value. Assume that we observe independent and identically distributed (i.i.d.) samples, which are either normal (Gaussian) distributed or large enough, the null hypothesis Assuming N − 1 degrees of freedom. This is the one-sample t-test.is approximately Student-t-distributed with yi could, for example, represent baseline activity that is measured in each repetition right before the presentation of the experimental stimulus. A natural null hypothesis is that the mean difference is zero, i.e., paired t-test, which replaces x by x − y in Equations (4), (5).A similar effect size is the mean difference x or y cannot be assumed to be normal distributed, a more appropriate test is the non-parametric Wilcoxon signed-ranked test which tests whether the mean population ranks differ  is a plot that visualizes the performance of such a binary classification system. It is obtained by plotting the true positive rate (TPR) against the false positive rate (FPR) when varying the threshold that divides the predicted condition into In many cases, one may be interested in quantifying the predictive accuracy of a binary classifier to separate experimental condition i et al.  for a coarea under the curve (AUC), and is equivalent to the probability that a classifier will correctly rank a randomly chosen pair of samples , where x is a sample from y is a sample from t-test discussed in section 2.2.2, if the data follow non-Gaussian distributions.A common way to reduce the ROC curve to a single quantity is to calculate the area beneath it Fawcett, . The resAssuming, without loss of generality, that higher values are indicative for class A = 0 and A = 1, while chance-level class separability is attained at A = 0.5. Thus, a common null hypothesis is H0 : A = 0.5.Perfect class separability is denoted by xn) the rank of xn  and VarH0(·) denote expected value and variance under the null hypothesis .A ≠ 0.5), the variances of U, W, and Â become data-dependent. The variance for general A can be approximated as  and U and W follow accordingly. A statistical test for the null hypothesis can be devised using thatwhere U and W).is approximately standard normal distributed for large sample sizes . Suppose that for each subject, we have N i.i.d. pairs of observations The H0 : ρ = 0. Assessing the statistical significance of Pearson correlations can be done using the Fisher z-transformation  has a bivariate normal distribution, then If , β1, …, βK are corresponding regression coefficients, β0 is an intercept parameter, and ηn is zero-mean, uncorrelated noise. In a neuroimaging context, the samples yn could represent a neural feature such as the activity of a particular brain location measured at various times n, while the xn,k could represent multiple factors thought to collectively explain the variability of yn such as the type of experimental stimulus or behavioral variables. In some fields, such a model is called a neural encoding model. It is also conceivable to have the reverse situation, in which the xn,k represent multiple neural features, while the dependent variable yn is of non-neural origin. This situation would be called neural decoding.where the dependent variable xn,k could be either categorial  or continuous. The specific case in which all independent variables are categorial is called analysis of variance (ANOVA). Linear models therefore generalize a relatively broad class of effect size measures including differences between class-conditional means and linear correlations  regression. The resulting estimate is also the maximum-likelihood estimate under the assumption of Gaussian-distributed noise. Using the vector/matrix notations y = Xβ + η. The OLS estimate is then given byThe most common way to estimate the regression coefficients βxn,k. The null hypothesis for factor k having no explanatory power is H0 : βk = 0. The estimated variance of The estimated coefficients where N − (K + 1) degrees of freedom. A similar procedure can be devised for regularized variants such as Ridge regression . Other quantities related to subject s are also indexed by the subscript s, while the same quantities without subject index denote corresponding group-level statistics.In the following, our goal is to combine the data of several subjects to estimate a population effect and to assess its statistical significance. We denote the number of subjects with Two different models may be formulated for the overall population effect.In the fixed-effect (FE) model, we assume that there is one (fixed) effect size θ that underlies each subject, that isThe observed effect s denotes the deviation of the subject's observed effect from the true effect θs = θ. We assume that the noise terms ϵs are independent, zero-mean random variables with subject-specific variance where ϵH0 : θ = θ1 = … = θS = θ0, where θ0 denotes the zero effect.The null hypothesis tested by a fixed-effect model is that no effect is present in any of the subjects. Thus, In the random-effects (RE) model, the true effect sizes are allowed to vary over subjects. They are assumed to follow a common distribution of effects with mean θ. The observed effect s denotes the deviation of the subject's observed effect from the true subject-specific effect θs, and where ξs denotes the deviation of the true subject-specific effect θs from the population effect θ. ξs and ϵs are assumed to be zero-mean, independent quantities. The subject-specific variance of ϵs is s is where ϵH0 : θ = θ0), while each individual subject-specific effect θs may still be non-zero.The null hypothesis being tested is that the population effect is zero  relying only on a fixed-effect analysis: We may arrive at spurious results, as the underestimated variance leads to p-values that are too low  studies, this approach is sometimes pursued when computing “grand-average” (group-level) waveforms of event-related potentials (ERP) that are elicited by the brain in response to a stimulus.p-values.Pooling the samples of all subjects may violate the assumption of identically distributed data underlying many statistical tests. Depending on the type of analysis, this may result in an over- or underestimation of the effect size, an over- or underestimation of the effect variance, and ultimately in over- or underestimated X and Y, are modeled for S = 4 subjects. N = 20 samples were independently drawn for each subject and variable from Gaussian distributions according to s = 1, …, S, n = 1, …, N, where the notation 2. The subject-specific offsets μs were independently drawn from another Gaussian: X and Y is correctly identified for each subject by Welch's two-sample t-test (p ≤ 0.02). Because of the substantial between-subject variance, this difference is, however, not significant in the pooled data of all subjects (p = 0.29). See Figure The following two examples illustrate the problem. In both cases, two variables, X and Y for each subject . However, the presence of subject-specific offsets μs causes a strong correlation of X and Y across the pooled data of all subjects , the test statisticwhich is an unbiased estimate of S − 1 degrees of freedom. This is the standard one-sample t-test applied to the individual effect sizes θ1, …, θS.is t-distributed with The naive summary-statistic approach is valid both under the fixed-effect and random-effects models  is valid regardless of the choice of the non-negative weights αs, s ∈ {1, …, S} as long as these weights are statistically independent of the corresponding effect size estimates. One popular choice is to assign equal weightsThe S data points on the group-level only. The equal-weighting approach instead uses the subject-level variances. That is, following Equation (35):to all subjects, such that S − 1 degree of freedoms. This translates into a power gain, as illustrated in the simulation presented in section 3. However, estimating the between-subject variance for a random-effects model is not straightforward, and also may introduce biases and variability (see section 2.6.3).If the individual Interestingly, the choice of equal weights is suboptimal in terms of obtaining a group-level effect size estimate s should have a lower impact on the overall estimate than those that are estimated with high confidence. Inserting into Equation (35), we obtain the optimal valueThis result is consistent with the intuition that less precise θNote, however, that, by using data-dependent weights, the inverse-variance-weighting approach may not always result in unbiased group-level effect size estimates. The potential implications of correlations between individual subject-level effect sizes and their variances are demonstrated in section 3.3 and further discussed in section 5.1.To perform inverse-variance weighting under the random-effects model, the between-subjects variance the between-subject variance nd Laird  is estimwhere p-values may become too small when the number of subjects S is small  are generally over-estimated  or weights that are proportional to the subjects' sample sizes  or specific versions derived for equal and inverse-variance weighting schemes in Equations (38) and (40). Finally, the estimated mean effect is subjected to a z-test as introduced in Equation (36).The sufficient-summary-statistic approach is summarized in Algorithm 1. First, the subject-level effect sizes θDifferent effect sizes and their corresponding variances have been discussed in section 2.2. With the exception of the Pearson correlation coefficient, these measures can be directly subjected to the inverse-variance-weighting approach. That is, H0, 1, …, HS0, , Stouffer's method aims to determine whether all individual null hypotheses are jointly to be accepted or rejected, or, in other words, if the global null hypothesis H0 :  is true. In general, the individual Hs0,  may not necessarily refer to the same effect size or even effect size measure, and the p-values for each individual hypothesis may be derived using different test procedures including non-parametric, bootstrap- or permutation-based tests. In the present context of nested data, Stouffer's method can be used to test group-level null hypotheses in the fixed-effect setting, i.e., the absence of an effect in all S subjects of the studied population.A general approach for combining the results of multiple statistical tests is Stouffer's method  and (46) into a group-level one-tailed p-value p1. The group-level two-tailed p-value is then obtained as p2 = 2·min (see Equations 1–3)  The data are generated either within a fixed-effect or a random-effects model. (2) The data are generated from either a Gaussian or a non-Gaussian distribution. In each scenario, we compare the methods' abilities to reject the null hypothesis when we vary the true class-conditional mean difference d.In the following, we present a set of simulations, in which we compare the statistical approaches reviewed above to test for a difference between two class-conditional means in artificial data. We consider two conditions S = 5 or S = 20 subjects s, s ∈ {1, …, S}, were generated as follows. First, subject-specific class-conditional mean differences ds were sampled according toData for rand = 0, while for the random-effects model, we set σrand = 0.2.where vs is uniformly distributed between 0.5 and 2, d, was varied across {0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3}.We then sampled H0rejection rate, which is the fraction of the test runs in which the null hypothesis was rejected. When the null hypothesis is true (d = 0), the H0 rejection rate is identical to the error or false positive rate of the statistical tests under study. In the converse case, in which the null hypothesis is false (d≠0), the rejection rate determines the power of the test. All statistical tests were performed at significance level α = 0.05. An ideal test would thus obtain a H0 rejection rate of 0.05 when the null hypothesis is true (d = 0), and a rejection rate of 1 otherwise. The higher the H0 rejection rate in the presence of an effect (d ≠ 0), the higher is the power of a test. However, if the null hypothesis is true, a H0 rejection rate greater than α indicates the occurrence of spurious findings beyond the acceptable α-level.All experiments were repeated 1,000 times with different samples drawn from the same distributions. We report the S = 5 and S = 20 subjects. The “pooling” approach consists of pooling the samples of all subjects and performing one two-sample t-test (cf. section 2.4). “Naive (paired t-test)” refers the naive summary-statistic approach, in which each subject's mean difference is treated as an observation for a group-level paired t-test (cf. section 2.5). Four variants of the sufficient-summary-statistic approach are considered (cf. section 2.6). These variants differ in assuming either random effects (RE) or one fixed effect (FE), and in using either the inverse-variance-weighting scheme (Equation 39) or equal weights (Equation 37). “Stouffer” finally refers to using Stouffer's method to combine p-values obtained from subject-level two-sample t-tests (cf. section 2.7). Note that all group-level tests are carried out two-tailed.Figure t-test.In line with our previous considerations, data pooling yielded very low power in the presence of an effect both under the fixed-effect and random-effects models. The highest power is achieved in both cases by the inverse-variance-weighted sufficient-summary-statistic approach, followed by Stouffer's method, the sufficient-summary-statistic approach using equal weights, and the paired Considerable differences are observed between the fixed-effect and random-effects settings. For data following the fixed-effect model, the fixed-effect variants of the sufficient-summary-statistic approach display only a negligible advantage over their random-effects counterparts, indicating that the latter succeed in estimating the between-subject variance to be zero. Moreover, in the case of equal class means, all approaches achieve a false positive rate close to the expected value of α = 0.05.S = 5 subjects, the null hypothesis is still rejected too often (9% for inverse-variance weighting). This is due to the variability in the estimate of the between-subject variance S = 20 subjects are available, the expected false positive rate of α = 0.05 is achieved.The situation is different for data following a random-effects model. Here, the fixed-effect variants of the sufficient-summary-statistic approach as well as Stouffer's method and the pooling approach display false positive rates that are between two and five times higher (26%) than what would acceptable under the null hypothesis. This problem is substantially alleviated by the random-effect variants of the sufficient-summary-statistic approach. Nevertheless, when data is only available from t-test of subject-wise means) achieves the expected false positive rate of 0.05 regardless of the number of subjects, and therefore represents a valid statistical test also in the random-effects setting.The naive summary-statistic approach (paired S = 20 subjects following either the fixed-effect model (top left panel) or the random-effects model (top right panel). For comparison, the results obtained on Gaussian-distributed data following a random-effects model are displayed in the bottom panel. Four different statistical tests are compared: (1) the random-effects inverse-variance-weighted sufficient-summary-statistic approach for the difference between class-conditional means, (2) the same test for the area under the non-parametric receiver-operating curve (AUC), (3) the naive summary-statistic approach in the form of a paired t-test between subject-wise means, and (4) its non-parametric equivalent, the Wilcoxon signed rank test. Note that for the naive summary-statistic approaches, the mean differences of each subject are treated as observations for a group-level paired t-test or Wilcoxon signed rank test, respectively.Figure The figure shows that, as for Gaussian-distributed data, the inverse-variance-weighted sufficient-summary-statistic approach achieves considerably higher statistical power than the corresponding naive summary-statistic approaches. Furthermore, non-parametric approaches achieve a higher power for non-Gaussian-distributed data than their parametric equivalents assuming Gaussian-distributed data. This difference is particularly pronounced for the better performing inverse-variance-weighted sufficient-summary-statistic approaches. The difference for the naive summary approaches is much smaller, because subject-level averages tend to be more Gaussian according to the central limit theorem. In contrast, parametric approaches have only a very minor advantage over non-parametric ones for Gaussian-distributed data. Note further that, when the Gaussianity assumption of the parametric approaches is violated, spurious results can, in theory, not be ruled out. However, such effects are very small here.H0-rejection rates for a negative correlation, implying that subjects with lower (negative) mean differences exhibit larger variability. In the inverse-variance-weighting approach, the influence of these subjects is down-weighted, which leads to an overestimation of the actual mean difference at the group-level. As a result, the number of false positive detections under the null hypothesis H0 : d = 0 dramatically increases. The right panel of Figure t-test are used.In the presence of dependencies between subject-level effect sizes and corresponding variances, the resulting group-level effect size may become biased if inverse-variance weighting is used. To demonstrate this adverse effect, we simulated data exhibiting a perfect correlation between the difference of the two group means and the standard deviation associated with this difference, represented by the square root of Equation (7). The between-subject variance was thereby kept at the same level as in the preceding random-effects simulations  by taking the subject-level variances into account.The bottom left panel of Figure t-tests and non-parametric AUC tests according to Equations (12) and (15). No substantial difference is found between the two except for a narrow time interval around 200 ms post-stimulus, in which the non-parametric test yields higher z-scores. Overall, this result suggests that the raw samples are approximately normal distributed, justifying the use of the parametric test.Finally, the bottom right panel of Figure t-test on subject-level effect sizes is valid, but has suboptimal statistical power. With the sufficient-summary-statistic approach and Stouffer's method, we discussed two general strategies that combine the simplicity and low complexity of “naive” approaches with higher statistical power by using prior knowledge about the distributions and variances of the subject-level effect sizes. The benefit of these two strategies over the “naive” approaches was demonstrated in a set of simulations. Note that the degree of improvement due to using sufficient summary statistics depends on the number of trials per subject vs. the number of subjects. Therefore, differing observations can be found in the literature  can be further combined into a higher-level average with known variance.Importantly, while we here only considered data with two nesting levels, both Stouffer's method and the sufficient-summary-statistic approach naturally extend to hierarchies with arbitrary numbers of levels. For example, t-test. An alternative is to use weights proportional to the subject-level sample sizes  may exhibit more variable activity representing their less constrained mental state. In practice, it is, therefore, advisable to test for monotonous relationships between effect size and variance, for example using Spearman's rank correlation. If no such relationship is found, the inverse-variance-weighted sufficient-summary-statistics approach can be used. In the opposite case, we recommend the use of the sufficient-summary-statistics approach using equal weights, which still improves upon the naive group-level The notions of “fixed” and “random” effects are used differently in different branches of statistics. See, for example, Gelman  for a diHere we define a fixed effect (FE) to be constant across subjects, while a random effect (RE) is allowed to vary across subjects. The fundamental model underlying RE analysis is given by Equations (28) and (29), while the FE model is defined in Equation (27). These definitions are used in the meta-analysis literature Field, , which cIn parts of the neuroimaging literature, a different interpretation of the fixed-effect model is predominant . These models extend the multiple linear regression model discussed in section 2.2.5 to deal with nested data. Following Hox et al. , this isThe subject-specific coefficients are further expressed ass0,  is a subject-specific intercept, L known subject-resolved independent variables zl, s, k, s, and εk, s is group-level zero-mean noise. In this complete form, all coefficients are subject-specific. We therefore speak of a random-effects nested linear model. It is also conceivable that only some of the coefficients are subject-specific, while others are shared between subjects. For example, in some applications it may be reasonable to model subject-specific intercepts βs0, , but identical effects βk, 1 = … = βk, S = βk for all subjects. A resulting model would be called a mixed-effects nested linear model.where γNested linear models are very general and allow for more complex statistical analysis than the procedures for estimating and testing group-level effects discussed here. On the downside, the estimation of nested linear models is difficult because no closed-form solution exists in the likely case that the variances of the subject- and group-level noise terms are unknown  are maintained except for the effect that is measured by θ. As such, surrogate data can provide an empirical distribution for p-values, which can be subjected to Stouffer's method to test for population effects under the fixed-effect model. Originally introduced in the context of identifying nonlinearity in time series  statistical inference is performed jointly for multiple measurement channels, the resulting group-level e et al. , Nicholse et al. , and Pere et al. .Another way to perform inference for multivariate data is to use inherently multivariate effect size measures such as canonical correlations, coefficients of multivariate linear models, the accuracy of a classifier (e.g., Haxby et al., t-test. Because the true single-subject classification accuracies can never be below chance-level, the group-level null hypothesis being tested is the fixed-effect hypothesis of no effect in any subject. Another problem with “information-like” measures is that certain confounds are not appropriately controlled for, because confounding effects of different direction do not cancel each other out (Todd et al., A distinction is made in the neuroimaging literature between so-called “activation-like” and “information-like” effect size measures. Allefeld et al.  argue thIn this paper, we have reviewed practical approaches to conduct statistical inference on group-level effects in nested data settings, and have demonstrated their properties on simulated and real neuroimaging data. With the sufficient-summary-statistic approach, we highlighted an approach that combines computational simplicity with favorable statistical properties. We have furthermore provided a practical guideline for using this approach in conjunction with some of the most popular measures of statistical effects.This work includes human EEG data from a previously published study (Haufe et al., Both authors contributed substantially to all aspects of the work including the conception and design of the work, the acquisition, analysis, and interpretation of data for the work, drafting of the work and revising it critically for important intellectual content. Both authors approved the final version to be published and agree to be accountable for the content of the work.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Biological networks consisting of molecular components and interactions are represented by a graph model. There have been some studies based on that model to analyze a relationship between structural characteristics and dynamical behaviors in signaling network. However, little attention has been paid to changes of modularity and robustness in mutant networks.In this paper, we investigated the changes of modularity and robustness by edge-removal mutations in three signaling networks. We first observed that both the modularity and robustness increased on average in the mutant network by the edge-removal mutations. However, the modularity change was negatively correlated with the robustness change. This implies that it is unlikely that both the modularity and the robustness values simultaneously increase by the edge-removal mutations. Another interesting finding is that the modularity change was positively correlated with the degree, the number of feedback loops, and the edge betweenness of the removed edges whereas the robustness change was negatively correlated with them. We note that these results were consistently observed in randomly structure networks. Additionally, we identified two groups of genes which are incident to the highly-modularity-increasing and the highly-robustness-decreasing edges with respect to the edge-removal mutations, respectively, and observed that they are likely to be central by forming a connected component of a considerably large size. The gene-ontology enrichment of each of these gene groups was significantly different from the rest of genes. Finally, we showed that the highly-robustness-decreasing edges can be promising edgetic drug-targets, which validates the usefulness of our analysis.Taken together, the analysis of changes of robustness and modularity against edge-removal mutations can be useful to unravel novel dynamical characteristics underlying in signaling networks.The online version of this article (10.1186/s12918-017-0505-2) contains supplementary material, which is available to authorized users. Robustness and modularity are key properties to understand complex dynamics in large-scale biological networks. The former means the capability of a network to maintain functioning against external and internal perturbations , and theIn this work, we tried to investigate the changes of the modularity and the robustness by edge-removal mutations in signaling networks. Through intensive simulations using a Boolean network model , 24, we G where V and A denote a set of nodes and a set of directed edges, respectively, we consider a partition P = {V1,  V2, …,  VM} of V = maxP M(P). Since it is difficult to optimize the partition, we computed the averaged modularity value of 30 trials of partitions optimized by an existing algorithm [In this study, we examined the modularity by using the method in a previous study  and it hlgorithm .G, V and A denote a set of Boolean variables and a set of ordered pairs of the Boolean variables called directed edges, respectively. A state of each vi ∈ V is 1 or 0 which represents on or off state of the gene, respectively. Then a state of a network G is defined as a vector of the states of all nodes. A directed edge  ∈ A has a positive (activating) or negative (inhibiting) relationship from vj to vi. Here, we used a nested canalyzing function (NCF) model [Ims and Oms between 0 and 1 uniformly at random.In this study, we employed a Boolean network model introduced in previous studies , 30 to iF) model  (see AddF) model  can be gF) model , and canF) model , 33. In G a Boolean network with a list of update-rules F = {f1, f2, …, fN}. Every initial state converges to an attractor which can describe diverse network dynamics such as multi-stability, homeostasis, and oscillation [α the attractor which the initial state s converged. The network is considered as robust against a perturbation at vi if the attractor is conserved and we herein considered an update-rule mutation which describes a scenario that F is changed to Im and Om are changed into 1 − Im and 1 − Om, respectively). This update-rule mutation may represent a deleterious change in the function of a protein or gene [γ(G) is defined as follows:Let illation , 35. Let or gene , and hav or gene . Then thS is a set of initial states (i.e. S = 2N), and I(∙) is a function which outputs 1 or 0 if the condition is met or not, respectively. Because ∣S∣ is a very large number, we used a sample subset S to calculate γ(G).where m1 and r1 be the modularity and the robustness of the wild-type network, respectively. Given a removal rate parameter n (%), the mutant network is constructed by simultaneously removing approximately n percent of a total number of edges from the wild-type network. Then let m2 and r2 be the modularity and the robustness of the mutant network. We defined the changes of the modularity and the robustness by the edge-removal mutations as (m2 − m1) and (r2 − r1), respectively. An illustrative example of the notion about the changes of modularity and robustness by edge-removal mutations is shown in Fig. This study focuses on how the modularity and the robustness of a network are changed by edge-removal mutations. Let www.cellcollective.org) [To investigate real signaling networks, we used three datasets of signaling networks: a T-LGL survival network (T-LGL)  consistiive.org) .We need to extensively simulate randomly structure networks to verify that the new findings in real networks are generally conserved. In this study, we employed a shuffling model to generate random networks , 18. GivDEG) means the sum of the degrees of both end nodes of the edge.Degree of a node means the number of links incident to the node in a graph. On the other hand, the degree of an interaction (v0 → v1 → v2 → … → vL − 1 → vL is an FBL of length L (≥1) if there exist links from vi − 1 to vi  with v0 = vL and vj ≠ vk for j, k ∈ {0, 1, …, L − 1} and j ≠ k. The number of FBLs of a link e denoted by NuFBL(e) means the number of different FBLs involving e.An FBL is a circular chain where nodes are not revisited except the starting and the ending nodes . It playEBEW) is defined as the number of shortest paths between pairs of nodes that run along an edge [Edge Betweenness , and the results are shown in Fig. In addition, the increase of the robustness was positively related to the removal rate. To examine the relationship between the changes of modularity and robustness values, we scattered them in the cases that the removal rate is 1% Fig.  and 2% . Intrigup-values < 0.001). That is to say, removing edges with a higher degree, EBEW, or NuFBL is more likely to increase the network modularity. These results can be relevant to previous results. For example, the edges with high betweenness values are most likely to lie between subgraphs [P-values < 0.0001 using t-test). In other words, the degree, the edge-betweenness and the number of FBLs were positively correlated with the change of the modularity whereas they were negatively correlated in the random networks. It means that those structural characteristics might be a vital factor in controlling both the changes of the modularity and the robustness.We showed that the changes of the modularity and the robustness are correlated when a network is subject to edge-removal mutations. To reveal structural characteristics to affect the changes of the modularity and the robustness, we investigated the correlations of each of the changes of the modularity and the robustness with each of three edge-based structural properties, DEG, NuFBL and EBEW (see Methods for the definitions) in T-LGL signaling network  order of the change of the modularity (resp. the robustness). Considering the distributions of the change of the modularity (resp. robustness), K was chosen to 20, 20, and 18  for T-LGL, STF, and HIV-1 networks, respectively. Then High-MI (resp. High-RD) denotes the union of the edges each of which was included in the modularity-increasing (resp. robustness-decreasing) top-K edges. Accordingly, we identified High-MI (High-RD) groups consisting of 22, 79, and 42 edges  in T-LGL, STF, and HIV-1 networks, respectively. Furthermore, we defined High-MI-incident (High-RD-incident) group which is a set of genes incident to an edge in the High-MI (resp. High-RD) edge group, and found the number of genes in the High-MI-incident (resp. High-RD-incident) were 29, 81, and 59  in T-LGL, STF, and HIV-1 networks, respectively. The topological distributions of High-MI and High-RD edge sets in T-LGL, STF, and HIV-1 networks are shown in Fig. In the previous subsection, it was shown that the change of the modularity is positively correlated with the degree, the edge betweenness, and the number of involved FBLs with respect to the removed edges whereas the change of robustness is negatively correlated with them. From these results, we hypothesized that the edges whose removal will increase the modularity or decrease the robustness tend to be centrally located in signaling networks. To validate this hypothesis, we first specified “Highly-modularity-increasing” (High-MI) and “Highly-robustness-decreasing” (High-RD) sets of edges as follows: We examined the changes of the modularity and the robustness over 5000 trials of edge-removal mutations with 1% removal rate, and collected top-We conducted Gene Ontotlogy (GO) enrichment analysis  using ClueGO tool  to invesK edge sets ranked by a decreasing order of the robustness change among 5000 trials of edge-removal mutations with 1% removal rate. We found that (JAK → STAT3), (IP3R1 → Ca), and (gp41 → CD28) showed the highest frequency in the T-LGL, STF, and HIV-1 networks, respectively. We hypothesized that these edges can be candidates of edgetic drug-targets, because they most frequently caused the highest decreasing robustness through removal mutations. To validate this, we surveyed some recent experimental studies. Regarding (JAK → STAT3) interaction of T-LGL network  interaction of STF network  was found to be the highest frequency interaction . It is included in PI3K/Akt/mTOR pathway [JAK → STAT3), (IP3R1 → Ca), and (PI3K → PIP3) in T-LGL, STF, HIV-1, respectively . Considering that the number of involved FBLs was shown to be associated with the functional importance of a node or an interaction, it implies that the found interactions can be promising drug-targets.We performed a case study to show an application for edge-based drug discovery. For every interaction in High-RD group, we examined the inclusion frequency of the interaction in top-ork Fig. , it was l cancer . Next, t pathway , which i pathway . AnotherThere have been many computational studies about the network robustness and modularity, whereas there are few studies on investigating the modularity change and the robustness change. Through extensive simulations, we found that both the modularity and the robustness increased on average in mutant networks by edge-removal mutations in this study. However, it was interesting that the changes of the modularity and the robustness were negatively correlated. Another interesting finding is that the changes of the modularity and the robustness are positively and negatively, respectively, correlated with each of the degree, the number of FBLs, and the edge betweenness of removed edges. These results were consistently observed in randomly structure networks. Additionally, we identified two sets of genes which are incident to the highly-modularity-increasing and the highly-robustness-decreasing edges, respectively, and observed that they are likely to be central by forming a large connected component. These two gene sets were enriched with different GO terms and the investigation on the reason why such GO terms are related to modularity and robustness will be a future study. Finally, we found that the highly-robustness-decreasing edge can be considered for promising edge-based drug-targets. Taken together, our results in this study can be useful to unravel novel dynamical characteristics of signaling networks.