Autism spectrum disorder (ASD) is a neurodevelopmental alteration characterized by social/communicative deficits, repetitive/stereotyped movements, and restricted/obsessive interests. However, there is not much information about whether movement alterations in ASD comprise modifications at the basic kinematic level, such as trajectory and velocity, which may contribute to the higher level of processing that allows the perception and interpretation of actions performed by others, and hence, impact social interaction. In order to further explore possible motor alterations in ASD, we analyzed movement parameters in the Valproate (VPA) animal model of autism. We found that VPA-treated rats displayed greater movement acceleration, reduced distance between stops, spent more time in the corner of the open-field arena, and executed a number of particular behaviors; for example, supported rearing and circling, with no major changes in distance and velocity. However, in the social interaction test, we found other alterations in the movement parameters. In addition to increased acceleration, VPA-rats displayed reduced velocity, increased stops, reduced distance/stop and lost the social/non-social area discrimination that is characteristic of control rats in acceleration and stops variables. Hence, even if prenatal VPA-treatment could have a minor effect in motor variables in a non-social context, it has a crucial effect in the capacity of the animals to adjust their kinematic variables when social/non-social context alternation is required. Autism spectrum disorder (ASD) is a multifactorial group of neurodevelopmental alterations of unknown etiology, in which genetic and environmental factors are involved . This coMotor alterations in ASD were described in the earliest studies of this pathology by Furthermore, animal models of autism have allowed us to explore the cellular and molecular underlying mechanisms and, to some extent, the associated motor alterations . In partIn order to explore possible motor alterations in the VPA-model related to independent free exploration or social interaction, we evaluated kinematic parameters in two habitually used behavioral tests for laboratory rats: the open field test (OFT) and the three-chamber social test (3C-ST). The first one is designed to evaluate locomotor activity and exploratory behavior without social demands and the second test specifically evaluates social interaction. In brief, the aim of our study was to determine whether the VPA autism model shows kinematic particularities and if they are influenced by social or non-social contexts.ad libitum access to food and water). On embryonic day 12.5 (E12.5) experimental rats received a single i.p. injection of Sodium Valproate , while the control group received a saline solution injection. On postnatal day 21 (PN21), the day of weaning, pups were separated from their dam and males were housed in groups of 3–4 littermates until P30, when behavioral tests were performed. We only used males in this study, considering that prenatal VPA treatment produces effects both in males and females, but with qualitative and quantitative differences , and a second batch composed of 8 control and 8 VPA-treated rats.A black OFT of 60 × 60 cm was used. At the beginning of the test, every animal was located at the center of the illuminated arena and allowed to freely explore; such behavior was recorded for 5 min. OFT was conducted in two batches of control and experimental rats. A first batch composed of 13 control and 7 VPA-treated rats (n = 8) and VPA treated (n = 11) rats. The apparatus consisted of three 30 × 22 cm adjoining chambers of transparent acrylic, connected by 10 × 10 cm doors. Because we used juvenile rats (PN30) we have used the apparatus dimensions for adult mice, which are similar in weight  to examine anxiety levels, as previously reported  in the s1 using the “threshold” and “analyze particles” functions to obtain the Cartesian position coordinates  of the rat’s “center of mass” in each video frame. For all analyzed video sequences, a visual verification of whole photograms was performed in order to correct the data, when necessary. Even though time consuming, this method guarantees no biases in point position data. Data were then transferred to an Excel worksheet  and each video data set was calibrated to the actual dimensions of the testing arenas. From these data sets we calculated the displacement between two consecutive frames and, using logical functions, extracted the frames in which the rats were moving and those in which the rats remained immobile. These two variables allowed us to obtain two parameters: “traveled distance,” as the cumulative sum of frame to frame displacement, and “effective velocity,” as the total distance divided by the time during which the rat was actually moving. We used the term “effective,” rather than “mean” velocity because we calculated the velocity only when the animal moved (excluding the time when it remained immobile). “Mean acceleration” was calculated as the mean value of frame to frame positive change of velocity, only counting the frames with displacements. The parameter “stops” corresponds to the number of times that a frame-series with displacements was interrupted by, at least, one frame of immobility, from another frame-series with displacements. The parameter “distance/stop” was obtained, as the total distance divided by the total number of “stops.” These analyses were performed for both the OF and 3C-ST behavioral tests.All videos were recorded with a web video camera , converted to the grayscale AVI format, and then processed by ImageJ 1.52p software 1 using tSpecific behaviors, such as “supported rearing” . Multiple comparisons in minute-by-minute analysis from OFT and 3C-ST-habituation and social/non-social areas- were analyzed by two-way ANOVA, followed by Tukey  or Sidak  post hoc tests. In all cases, a p-value lower than 0.05 was considered as statistically significant.Statistical analyses were performed using GraphPad Prism 8.0.2 . The non-parametric two-tailed Mann-Whitney In order to analyze possible motor alterations in our VPA model, we decided to explore changes in kinematic parameters in two behavioral tests; one designed to evaluate locomotor activity (OFT), and another one usually chosen to evaluate social interaction (3C-ST).2), which was higher  than that of the control group (81.3 ± 5.8 cm/s2)    . Our ana U-test) . The tot U-test) .p < 0.0001, F = 14.04], without an effect of the VPA treatment , or minute × VPA interaction , thus evidencing a reduction of the same magnitude for both control and VPA-treated groups . Similarly, analysis in the VPA-group showed a “traveled distance” in the first minute of 517.9 ± 36.8 cm, which significantly decreased in the last minute, reaching 200.3 ± 29.8 cm . Regarding the behavior of “effective velocity” along the five minutes of the OFT, we did not observe significant effects by two-way ANOVA analysis for time , VPA treatment  nor interaction time × VPA . Control and VPA-groups showed a nearly invariable “effective velocity” along the trial period   = 3.643], VPA treatment  and time × VPA interaction . “Mean acceleration” significantly decreased  from the first minute (165.8 ± 14.8 cm/s2) to the last minute (106.9 ± 13.8 cm/s2), reaching values that were not statistically different from those of the control group in the last 2 min  to the end (minute 5). In fact, two-way ANOVA showed a significant main effect of time , no significant effect of VPA treatment  and significant time × VPA interaction . Even if no statistically significant difference was found between the control and VPA-group at any minute, the first minute in the control was statistically different from minute five in the same control condition .As shown in p = 0.0002, F = 6.892], a significant effect of VPA treatment  and significant time × VPA interaction . There was no statistically significant difference between the control and VPA-group in any minute, but the first minute in the control group was statistically different from minute five in the same control condition .On the other hand, minute-by-minute “distance/stop” showed a profile inverse to that of stops, which was expected considering no changes in traveled distance . “DistanThese data evidenced that prenatal VPA treatment specifically affects the acceleration of movements without influencing the “traveled distance” and “effective velocity” parameters. In addition, habituation to the OFT arena occurred normally at the level of “traveled distance” and only the VPA-group showed an habituation-like behavior at the level of “mean acceleration,” suggesting that not all kinematic parameters show a similar variation as a function of time. Interestingly, the parameters “stops” and “distance/stop” showed a constant behavior in the VPA-group, while the control group was able to modulate them along the time of the test, suggesting that VPA animals decrease in-time acceleration by decreasing the traveled distance but not increasing the stops.p < 0.01, Mann-Whitney U-test), without a significant effect on “returns.” Circling activity increased nearly three times in the total OFT arena (regardless of direction) in the VPA-group  and was more notorious in the OFT corners    lower time (2.41 ± 1.9%) compared to control group (7.99 ± 3%)  . In addi U-test) . Regardi99 ± 3%) .p = 0.43, Mann-Whitney U-Test)  . A remarp < 0.05, Mann-Whitney U-Test) demonstrating a higher level of anxiety in VPA-treated rats.Since anxiety is a fundamental sign in ASD individuals and in animal models of autism, we conducted the EPM test in order to corroborate this characteristic. 2) significantly increased in the VPA group (126 ± 18.5 cm/s2)   .p < 0.01, Mann-Whitney U-test)   . Inverse U-test) .p < 0.0001, F = 19.25], without an effect of VPA treatment  or time × VPA interaction . We observed a statistically significant gradual decline for “traveled distance” from the first to the last minute in control and VPA groups  = 22.02], no effect of VPA treatment , and significant minute × treatment interaction . There were no statistical differences between the control and VPA groups in each minute, but a statistically significant decline in the control group  and the VPA-group   . This be minute) .p < 0.0001, F = 11.97], VPA treatment  and time × VPA interaction . “Mean acceleration” was relatively stable during the 5 min of the 3C-ST habituation stage in the control group. On the other hand, the VPA-group showed a statistically significant decline  from the first minute (147.9 ± 18.8 cm/s2) to the last minute (98.1 ± 16.8 cm/s2), reaching values that were not statistically different from that of the last minute in the control group  = 2.009], a significant effect of VPA treatment  and a significant time × VPA interaction . Minute five of the VPA-group was significantly lower than the first minute in the same group .Regarding the behavior of “stops” during the 3C-ST habituation stage, we observed that the stops increased in the VPA-group compared to the control group in the first minute and a gradual decline to converge to control values was produced in minute five . Two-wayp = 0.1117, F = 2.009], a significant effect of VPA treatment  and a significant time × VPA interaction . The differences between the first minute and the last minute in control and VPA groups was statistically significant .The “distance/stop” parameter showed an inverse pattern respect to the “stops.” We observed decreased “distance/stop” in the VPA-group compared to the control group in the first minute and a gradual decline -also observed in controls-, to converge to control values in minute five . Two-wayIn general, the results obtained in the habituation stage of 3C-ST validate our analysis performed with the OFT, and show that prenatally VPA-treated rats do not display considerable differences in their kinematic characteristics (such as distance and velocity). However, a general increase in mean acceleration was present in all conditions, along with increased halts during trajectories and reduced distance between stops, more or less evident depending on the behavioral test used. In fact, increased “stops” and reduced “distance/stop” were more evident in the beginning of the OFT and habituation stage of the 3C-ST.p < 0.001; Mann-Whitney U-test)  . Hence, p = 0.064; Mann-Whitney U-test, p < 0.05; Mann-Whitney U-test, U-test); from 116.3 ± 3.8 cm/s2 in the control group to 197.7 ± 20.5 cm/s2 in the VPA-group   . These rp = 0.4944, F = 0.4923], but a significant effect of treatment , without area × treatment interaction . The VPA-group showed a reduced “traveled distance” in the social area compared to the control group , while no significant difference was found in the non-social area   or VPA groups . These results suggest that VPA-treated rats reduce their locomotor activity in the new condition; however, this parameter did not differentiate between social and non-social areas, in both control and VPA groups.Furthermore, the analysis of kinematic parameters to separate social from non-social areas showed a differential effect of VPA treatment. First, two-way ANOVA analysis of “traveled distance” showed no effect for area  and VPA treatment , without area × treatment interaction . A statistically significant reduction was observed in the VPA-group in social (p = 0.0153) and in non-social (p = 0.0004) areas, in relation to the control group (Sidak post hoc test) , while the VPA group showed an “effective velocity” of 8.05 ± 1.8 cm/s in the social area and 13.42 ± 2.34 cm/s in the non-social area . Therefore, “effective velocity,” in contrast to “traveled distance” is a parameter that discriminates between social and non-social contexts in both control and VPA-treated rats.In relation to “effective velocity,” two-way ANOVA analysis showed a significant main effect of area  and VPA treatment , without area × VPA interaction . A statistically significant increase of “mean acceleration” was observed in the VPA-group in both social (p < 0.01) and non-social (p < 0.05) areas, in relation to the respective areas in the control group (Sidak post hoc test) , but not in the VPA-group (p = 0.8200) . Two-way ANOVA analysis indicated a main effect of area , no effect of VPA-treatment  and a significant area × VPA interaction  (p = 0.0007), but not in the VPA group . However, no significant difference was observed between control and VPA groups in social (p = 0.1368) or in non-social (p = 0.0944) areas (Sidak post hoc test). This result suggests that the control group is able to discriminate social from non-social areas in this parameter, while the VPA-group loses that ability.“Stops” in social and non-social areas also showed a differential behavior between control and VPA groups. Two-way ANOVA analysis of stops indicated a main effect of area [= 8.326] . A statip < 0.0001, F = 32.29] and VPA treatment  and a significant area × VPA interaction  (p < 0.0001), but not for the VPA-group . No significant difference of distance/stop was observed in the social (p = 0.0940) area, but a significant difference was found in the non-social area when comparing the control with the VPA-group , suggesting that this parameter discriminates social from non-social areas in controls, but not in VPA treated rats.On the other hand, two-way ANOVA analysis of “distance/stop” showed a main effect of area [= 10.55] . The difp = 0.16; Mann-Whitney U-test, p = 0.64; Mann-Whitney U-test, p < 0.05; Mann-Whitney U-test, p = 0.030, Mann-Whitney U-test), suggesting that controls discriminate by context, an ability lost in the VPA-group. Regarding “distance/stop,” no statistical difference was found between control and VPA groups   .The results obtained in the social stage of the 3C-ST corroborated increased acceleration in the VPA group and suggests that control animals reduce their movement acceleration and increase their halts when they move in the social chamber, while VPA-treatment abolishes the ability to modulate acceleration and halts when the rats face social or non-social contexts.We have studied basic kinematic parameters in the juvenile rat VPA-model of autism using two behavioral tests (OFT and 3C-ST), which represent two different contexts. The most remarkable observation of our work was evidenced in the social stage of the 3C-ST, where, according to the “mean acceleration” and “stops” parameters, the social/non-social context discrimination was lost in the VPA-group, whereas distance does not discriminate a social context, while velocity does in both groups. In addition, we showed that VPA treatment produced consistent increases in “mean acceleration” and reduced “distance/stop,” with almost null effect on “traveled distance” and “effective velocity” in the OFT, an effect corroborated during the habituation stage of the 3C-ST. However, in the last case, we observed additionally increased “stops.” These results suggest that in a socially neutral context, the motor alterations in ASD may specifically affect certain kinematic parameters; i.e., acceleration and related variables as “stops” and “distance/stop,” without evidently affecting distance and velocity. Moreover, in a condition where there is an alternation of social vs. non-social contexts, velocity is also affected and more importantly, we evidenced an effect in the ability to dynamically change “mean acceleration” and “stops” parameters between social and non-social contexts. To our knowledge, this is the first report in an animal model of autism that shows this type of discriminative behavior for acceleration and stops variables in control rats, and its loss in VPA-treated rats.In the OFT, control and VPA-treated rats exhibited similar kinematic characteristics, such as total displacement and velocity. In addition, according to the “traveled distance” parameter, both groups seemed to equally habituate to the arena. In order to obtain a real parameter of movement velocity in general, but not associated with a particular type of movement , we calculated “effective velocity” as the mean velocity, considering only the time in which the animal showed a displacement of its center of mass, respect to the precedent video frame. Furthermore, despite the similarity displayed by control and VPA groups in these two basic kinematic parameters, we found an important difference in movement acceleration, measured as “mean acceleration.” Nonetheless, it is important to mention that we cannot infer whether these differences are associated with a particular movement characteristic; in other words, related to increased acceleration, which might represent less smooth movements or faster movement initiation. Even though a more precise kinematic study is necessary to clear this point out, we can conclude that the observed differences represent an intrinsic characteristic of the animal movements affected by the VPA treatment. A previous report in which kinematic parameters were studied in different strains of control mice has indicated that the acceleration-influenced parameter “velocity/stops,” may differentiate mice strains, and this parameter was associated with “darting behavior,” which can reflect vulnerability to stress, anxiety or jitteriness . In factOf note, VPA-treated animals normally habituate the “traveled distance” parameter in the OFT and the 3C-ST, while “mean acceleration” reflects a habituation-like behavior only in VPA animals. In addition, velocity showed a habituation-like behavior in both control and VPA rats in the little arena of habituation stage of 3C-ST. We suggest that acceleration is a more sensitive parameter for detecting anxiety in VPA animals. In fact, Since our kinematic parameters did not characterize any particular type of movement, we also evaluated walking, rearing, turns and circling behavior. When comparing control and VPA rats, we did not find any differences in walking velocity, but we had difficulties in finding video sequences in which VPA rats walked in a straight line, reflecting their propensity to perform circling behavior, especially in the corners of the arena test, where they also spent more time in comparison to controls. Increased circling behavior and supported rearing may suggest anxious behavior but, considering that standing upright is a normal behavior , we cannpostmortem encephalic tissue from ASD individuals , located in the upper brainstem, is fundamental in defining walking velocity , althougividuals . Moreoveividuals .Based on the exploratory component of our behavioral tests and taking into account the particular exploratory behavior displayed by VPA-treated rats in the OFT arena, basal ganglia circuits may be interesting regions in which the neurobiological underpinnings of the kinematic alterations observed in ASD could be further investigated. In fact, multiple alterations have been described in basal ganglia circuits in ASD individuals and animal models of autism . This coKinematic studies in ASD are scarce but, an interesting work showed a characteristic kinematic signature regarding micro-movements during a reaching task in ASD . The autAn interesting recent report of On the other hand, studies in the VPA-model do not normally address kinematic parameters, such as velocity and acceleration. Only a recent report evaluated velocity in control and postnatally VPA-treated rats in a context of empathy-like pro-social behavior, evaluating the willingness of rats to open a restrainer where a conspecific was trapped . InteresOn the other hand, atypical kinematics has been evidenced in ASD individuals by studying a simple sinusoidal arm movement and, importantly, the degree of such atypicality correlates with the ADOS score and the difficulty to interpret biological movement . Neverthbona-fide” model of autism- on the social/non-social context discrimination by the kinematic variables, acceleration and stops, suggesting that these specific parameters may reflect behavioral inflexibility, a fundamental alteration of ASD, pervasive even at the most basic levels of motor control.Finally, we provide, for the first time, evidence of the effect of prenatal exposition to VPA -a “ealiaga@ucm.cl.The datasets presented in this article are not readily available because data sharing is not applicable to this article as no new data were created in this study. Requests to access the datasets should be directed to EA, The animal study was reviewed and approved by the Ethic Committee of the Chilean Science and Technology National Commission (CONICYT).EA and JF designed the study, analyzed and interpreted the data, and wrote the article. NQ, SS-O, and FP performed the experiments, analyzed the data, and participated in the final correction of the article. FA, KM-G, and CM-R participated in the analysis and interpretation of data and editing of the article. All authors gave approval of the final version to be published and agreed to be accountable for all aspects of the work.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The Editors-in-Chief have retracted this article (Yang et al. All authors agree to this retraction.


Scientific Reports 10.1038/s41598-020-74174-x, published online 14 October 2020Correction to: The Supplementary Information published with this Article contains errors.The degree of Takashi Oka, “PhD., DMSc”, is incorrectly given as “PhD”.In addition, the excitation wavelength measurements within the “Supplemental materials and methods” section is omitted.Lastly, the explanation of the Figures given in the legend of Figures The correct Supplementary Information file is linked to this correction notice.Supplementary Information


THIS ARTICLE USES WORDS OR LANGUAGE THAT IS CONSIDERED PROFANE, VULGAR, OR OFFENSIVE BY SOME READERS. The presence of a significant amount of harassment in user-generated content and its negative impact calls for robust automatic detection approaches. This requires the identification of different types of harassment. Earlier work has classified harassing language in terms of hurtfulness, abusiveness, sentiment, and profanity. However, to identify and understand harassment more accurately, it is essential to determine the contextual type that captures the interrelated conditions in which harassing language occurs. In this paper we introduce the notion of contextual type in harassment by distinguishing between five contextual types: (i) sexual, (ii) racial, (iii) appearance-related, (iv) intellectual and (v) political. We utilize an annotated corpus from Twitter distinguishing these types of harassment. We study the context of each kind to shed light on the linguistic meaning, interpretation, and distribution, with results from two lines of investigation: an extensive linguistic analysis, and the statistical distribution of uni-grams. We then build type- aware classifiers to automate the identification of type-specific harassment. Our experiments demonstrate that these classifiers provide competitive accuracy for identifying and analyzing harassment on social media. We present extensive discussion and significant observations about the effectiveness of type-aware classifiers using a detailed comparison setup, providing insight into the role of type-dependent features. Disclaimer: This article uses words or language that is considered profane, vulgar, or offensive by some readers. Owing to the topic studied in this article, quoting offensive language is academically justified but neither we nor PLOS in any way endorse the use of these words or the content of the quotes. Likewise, the quotes do not represent our opinions or the opinions of PLOS, and we condemn online harassment and offensive language.Although social media has enabled people to connect and interact with each other, it has also made people vulnerable to insults, humiliation, hate, bullying–facing threats from individuals who are either known  or unknown . Please note that in this work, cyberbullying and harassment are used interchangeably. A Pew research study reports  that oneHate speech is “speech that denigrates a person because of their innate and protected characteristics” . Th. Thone gficient)  metric wficient) . Convent“I” is prevalent in the sexually non-harassing corpus with e = −1.2, which is highly significant, e.g., i’m le***an kiss. Furthermore, the feature “MONEY” is prevalent in the harassing corpus with e = 2.9. E.g., send me free money b**ch h*es i won’t give you anything to dance to you h*e a*s industry b*t*h d*cks**king p*r*star people.The pronoun “YOU” is prevalent in the harassing corpus with e = 0.9, e.g., Vishalp sikanda, Quideazam hahahaha u p**i can block u cant debat u p**i I***an. The “COMPARATIVE” feature is prevalent in racial non-harassing corpus with e = −0.84, e.g., save block p**i like po yung comment ni richard fronda (the word ‘like’ is an indicator of comparison in LIWC). Thus, these features can be used to discriminate between harassing and non-harassing tweets.The pronoun “SHE” and “HE” with e = −0.9 and the pronoun “WE” with e = −0.8 are prevalent in the non-harassing corpus, e.g., realdonaldtrump putin a**hat just like word can express displeasure leader god help us (us indicates the pronoun ‘WE’). The “RISK” feature is significant in non-harassing with e = −1.9, e.g., f*** wrong democratic senators. The word ‘wrong’ represents a risk feature in LIWC dictionary. Other sample risk related words are ‘danger’, ‘doubt’, etc. Furthermore, the “ANXIETY” feature with e = −0.92 is significant in the non-harassing corpus. E.g., well i’m true dumb f*** democrat wouldn’t doubt.The pronoun “NEGATION” with e = 2.3 is prevalent in the harassing corpus (probably because of the negative language used for referring to the body and appearance-related subjects). E.g., Taylor swift cant shake c**el toe. The other significant feature in the harassing corpus is the “PAST TENSE”. E.g., Ugli a** didn’t go run yesterday get work f*t**s. Furthermore, the “COMPARATIVE” feature is prevalent in appearance-related harassing corpus with e = 0.63. E.g., hey lardass notice your look pizza perhaps like f*** salad a**hole. The word ‘like’ indicates a comparative feature.“FEMALE REFERENCE” feature with e = 2.3 is highly significant in intellectual harassing corpus  E.g., She is dumb f***.The “DISCREPANCY” with e = −1.5 is prevalent in the non-harassing corpus e.g., boss brought drunken sugar cook explain there alcohol just sh**face.Q1: whether or not offensive words are observed as frequent words, Q2: whether or not the frequent words in harassing corpora differ from those in non-harassing corpora, and Q3: whether or not frequent words are type-sensitive, in other words, whether the frequent words vary with type of context. We investigate the relationship between the offensive words employed in collecting our corpora and the specific lexical items in the crawled corpora. We determine Regarding Q1, as expected, we observed that offensive words are commonplace in both harassing and non-harassing corpora across types cf. Figs  and 4. Iper se is not necessarily harassing.To reply quantitatively to Q3, we ran an annotation task on the top-15 most frequent words for each type of harassing corpus as well as the corresponding non-harassing corpus. The description of this task is as follows: we asked the human annotators  to determine whether or not a given frequent word is related to the associated type either explicitly or implicitly. E.g., the words “eat” or “food” are implicitly related to appearance while they seem far from the type racial. The results of this exercise appear in One caveat is that the most frequent words appearing in the sub-corpus associated with each type are predominantly stop-words or curse words, as our initial seed terms are biased to an offensive lexicon. Ignoring these words, whose presence cuts across different types of harassment, revealed that the following prominent word groups are associated with various harassment types, shedding light on the possible features that may elicit harassment: (i) In the appearance-related harassment corpus, target words such as “eat”, “ugly”, “fat”, “gym”, and “weight”, are present. (ii) In the intellectual harassment corpus, target words such as “dumb”, “stupid”, “work”, and “head”, are present. (iii) In the political harassment corpus, the target words such as “realdonaldtrump”, “libtard”, “dumb”, “touch bag”, “stupid”, and “cnn”, are present. (iv) In the racial harassment corpus, target words such as “m*ki”, “n**ger”, “b**ner”, “ch**k”, “muslim”, “i**ian”, “moron”, and “jew”, are present. (iv) In the sexual harassment corpus, target words such as “hump”, “hussy”, “l**k”, and “grab”, are present.We aim to develop effective supervised learning methods to detect harassing language automatically and distinguish it from non-harassing language for each contextual type. The state-of-the-art contains various approaches for detecting harassing content from non-harassing content but not for discriminating the type of harassment. We approach this gap in two ways. The first is to build individual binary classifiers that identify a particular type of harassment, e.g., a binary classifier that identifies only racial content or a binary classifier that classifies just offensive political content. The second approach uses the state-of-the-art methods to detect harassing language; after such recognition, we can employ a type-aware classifier to predict the associated type for that harassment incident. We implemented both approaches. Initially, we trained the individual classifiers for each type. In another approach, we built up a binary classifier that differentiates harassing content from non-harassing content regardless of their type. Note that any classifier from the state-of-the-art can substitute for this part. Then, we built up a multi-class classifier that predicts the type of harassment incident. The results of our experiments for both approaches reveal high accuracy. Furthermore, to verify the effectiveness of our classifier, we apply transfer learning by running our classifier on the Golbeck corpus and assess its performance for how successfully it predicts the type of harassment. In the following, we present the details of our experiments.We utilized four approaches for transforming tweets to numerical representations : (i) the conventional vectorization approach TFIDF, (ii) word2vec, (iii) fastText and (iv) a LIWC vector. We feed our classifiers with each of these individual vectors or a combination of them.We use this approach  to transwk (i.e. context word) appearing in the neighborhood of the context word wi (i.e. target word), Distributional semantics   play a v Corpora  leveragi Corpora  as the u Corpora , 57. OurThe vector obtained by running the LIWC tool is denoted by L.As the number of harassing tweets is not equal to the number of non-harassing ones in our corpus—in fact, it varies for each type—we prepared balanced datasets for training the classifier. We prepared five type-aware training data sets using an under-sampling approach taking all of the harassing tweets with an equal number of non-harassing (randomly sampled). Also, we prepared a combined training data set considering all of the harassing tweets regardless of their type and an equal number of non-harassing tweets. Support Vector Machine (SVM) [K-Nearest Neighbors (KNN) [Gradient Boosting Machine (GBM) [Naive Bayes (NB) [In our experimental study, we trained four types of classifiers, using (i) ne (SVM) , (ii) K-rs (KNN) , (iii) Gne (GBM) , and (ivyes (NB) . We relyTo gain insight over the effectiveness of various features, we feed the GBM classifier with various feature settings. The fine-grained results of our experiment are listed in We also trained a binary classifier on our combined corpus where it can differentiate the harassing language from non-harassing regardless of the contextual type. In situations that the type of harassment does not play a role, or type detection must occur after the harassment detection, using such a generic classifier is necessary. Apart from building binary classifiers for predicting types, we trained a multi-class classifier to predict the type of harassment incidents. We trained several multi-class classifiers, among them the GBM classifier outperformed others. Herein we report the result for GBM classifier only. We used W(S)+F(S) vectorization approach as the input feature. Then, we trained this classifier on a corpus containing all of the sub-corpora from the previous step. This corpus has samples with six various labels where five labels indicate a particular type of harassment and the last label indicates “non-harassing” implying there is no harassing language. Since this work was the first to introduce contextual type for harassment, comparison to the state-of-the-art that relies only on two or three variants of harassment, is unfair. However, to verify the effectiveness of our type-oriented multi-class classifier, we tested it on the harassing tweets from the Golbeck corpus  that is a publicly available state-of-the-art harassment-related corpus . This co@usr you deserved to be raped by a thousand Muslims in your c**t a**hole, our classifier classified that as sexual harassment and not racial because of the word ‘rape’. Similarly, the tweet @usr @usr lol it’s not against women. It’s against f***ing feminist c***s like you. #feminazi #womenagainstfeminism was classified as sexual. Such cases are ambiguous because even manual annotation is highly subjective. In other words, categorizing harassment is highly subjective and the boundary between types is not rigid. In majority of the overlapping cases , the tweets were classified as sexual rather than racial. We also analyzed errors in political tweets and concluded that harassment signal can be: (i) implicit, e.g., John Boehner blames Democrats for #shutdown. He better stop drinking cuz a few more drinks and he starts blaming the J*ws f, (ii) ambiguous ??? You’re a wh*** to the telecom industry, i hope your constituents vote you out., (iii) unreliable, e.g., It’s going to be a republican government in the US next term. Democrats can kiss their presidency bid goodbye. Let the J*ws rule!, (iv) poorly captured through annotation, e.g., the tweet @TrueNugget @FeministPeriod @OregonState Man college is becoming more and more a mistake. in the Golbeck corpus. Our classifier misses them as they are weak cases of harassment.To make sense of classifier errors, we examined a couple of tweets classified as sexual. E.g., for Our project involves analysis of Twitter data that is publicly available and that has been anonymized. It does not involve any direct interaction with any individuals or their personally identifiable data. So our work does not meet the Federal definition for human subjects research, specifically, “a systematic investigation designed to contribute to generalizable knowledge” and “research involving interaction with the individual or obtains personally identifiable private information about an individual”. Thus, this study was reviewed by the Wright State University IRB and received an exemption determination.In this paper, we introduced five contextual types for harassment, namely, (i) sexual, (ii) racial, (iii) intellectual, (iv) appearance-related and (v) political. We presented experiments with a type-aware tweets corpus to analyze, learn, and understand harassing language for each type. Our contribution lies in providing a systematic and comparative approach to assessing harassing language from linguistic and statistical perspectives. Furthermore, we built type-specific classifiers, and the results of our experiments show the importance of considering the contextual type for identifying and analyzing harassment on social media.In general, a single tweet identified as “harassing” may not provoke the same intense negative feeling that we associate with that word in the real-world scenario. However, in practice, “conversational” exchanges containing a sequence of such tweets can rise to the level of harassment causing mental and psychological anguish, and fear of physical harm. Nevertheless, our current Twitter dataset is limited to annotating single tweets in isolation for harassment. Furthermore, the reliable assessment of the type of harassment is a difficult problem because it requires significant knowledge of current events and common-sense. We plan to extend this work by learning the language of harassers as well as victims, and further study the contribution of non-verbal cues  for identifying online harassment activities, particularly on social media.


Modeling and simulating movement of vehicles in established transportation infrastructures, especially in large urban road networks is an important task. It helps in understanding and handling traffic problems, optimizing traffic regulations and adapting the traffic management in real time for unexpected disaster events. A mathematically rigorous stochastic model that can be used for traffic analysis was proposed earlier by other researchers which is based on an interplay between graph and Markov chain theories. This model provides a transition probability matrix which describes the traffic’s dynamic with its unique stationary distribution of the vehicles on the road network. In this paper, a new parametrization is presented for this model by introducing the concept of two-dimensional stationary distribution which can handle the traffic’s dynamic together with the vehicles’ distribution. In addition, the weighted least squares estimation method is applied for estimating this new parameter matrix using trajectory data. In a case study, we apply our method on the Taxi Trajectory Prediction dataset and road network data from the OpenStreetMap project, both available publicly. To test our approach, we have implemented the proposed model in software. We have run simulations in medium and large scales and both the model and estimation procedure, based on artificial and real datasets, have been proved satisfactory and superior to the frequency based maximum likelihood method. In a real application, we have unfolded a stationary distribution on the map graph of Porto, based on the dataset. The approach described here combines techniques which, when used together to analyze traffic on large road networks, has not previously been reported. In the past decade, research and development of smart city applications have become an active topic , 2. ThesRobocar City Emulator (RCE). We extract geographical information from OpenStreetMap (OSM) and transform this data into a routing map graph. The simulation takes place on a rectangular part of the OSM. The traffic simulation model of the RCE is based on the Nagel-Schreckenberg cellular automata model 63].V is the vector π ≔ (πv)v∈V where πv ≥ 0 for all v ∈ V and ∑v∈Vπv = 1. That is a p.d. π is a normalized πv as the proportion of the number of vehicles which drive through the crossing v with respect to the whole number of vehicles in the city at a fixed time period. A Markov kernel or transition probability matrix on V is defined as a real kernel P ≔ (puv)u,v∈V such that puv ≥ 0 for all u, v ∈ V and ∑v∈Vpuv = 1 for all u ∈ V, i.e., pu ≔ (puv)v∈V is a p.d. on V for all u ∈ V. The quantity puv ∈  is called the transition probability from vertex u to vertex v. The kernel P is said to be G-subordinated if puv > 0 for a pair u, v ∈ V implies  ∈ E or u = v, i.e., P as a matrix on V is G-subordinated in the weak sense. It is well known, see [P on V, an associated digraph GP =  can be introduced in the following way: for a pair u, v ∈ V   ∈ EP if and only if puv > 0. Thus, P is G-subordinated if and only if EP ⊆ E ∪ S, i.e., GP is the subgraph of the digraph G extended with its loops S. In other words, a G-subordinated Markov kernel P is a stochastic matrix on V with support E ∪ S. Then, the sum condition for a G-subordinated Markov kernel P can be rewritten as:p00 = 0).A probability distribution (p.d.) on own, see , that foπ on V is a stationary distribution (s.d.) of the kernel P if ∑u∈Vπupuv = πv for all v ∈ V. For a G-subordinated Markov kernel P this formula, the so-called global balance equation, can be expressed as:A p.d. The stationary distribution can be derived by solving the linear P on V and the strongly connected property of the physical road network G if the Markov and graph structures are compatible with each other. The Markov kernel P on V is called G-compatible if, for any u, v ∈ V such that u ≠ v, puv > 0 if and only if  ∈ E. Note that the G-compatibility implies the weak G-subordination for a Markov kernel P, however the converse is not true.We show that there is a direct connection between the uniqueness of s.d. of a Markov kernel P is G-compatible then the strong connectivity of G implies that the associated graph GP to the Markov kernel P is also strongly connected. In this case, the Markov kernel (the transition matrix) P is called irreducible. Thus, by Theorem 1 in [Clearly, if rem 1 in , see alsrem 1 in  the follTheorem 1. If a road network G is strongly connected then there is a unique stationary distribution π to any G-compatible Markov kernel P. Moreover, this distribution satisfies πv > 0 for all v ∈ V.The main consequence of this theorem is that, in case of any physical road network augmented by the ideal vertex 0, all of the Markov kernels defined on the road network that has positive transition probability on all roads have unique stationary distribution. Thus, it is reasonable to suppose that a real traffic which follows a Markovian dynamic has a local unique stationary distribution in a short time period that can be explored by observing the traffic.V-valued random variable (r.v.) is a X: Ω → V measurable function, i.e., v ∈ V. In this case, X is a random function on the set V of vertices. For example, X can be the random position of a vehicle on the road network G, where the position refers to the actual vertex which the vehicle belongs to. Then, v ∈ V. By v ∈ V, a r.v. X induces a p.d. πX on V.Let V-valued r.v.’s is a Markov chain on the state space V if the Markov property holds:v0, …, vt ∈ V. If X, X′ are V-valued r.v.’s then for the conditional distribution P = (pvv′)v,v′∈V, v, v′ ∈ V, we shall also use the notation X|X′. Clearly, X|X′ is a Markov kernel on V.A sequence The main concepts of this paper are the Markov random walk and the Markov traffic defined in the following way.Definition 2. Let the road network G be strongly connected and let P be a G-compatible Markov kernel on V with unique s.d. π. Moreover, let V such that Xt|Xt−1 ∼ P for all Markov random walk on the road network G with Markov kernel P.k .The set π of fv = 1 for a fix v ∈ V and 0 otherwise, see page 75 in [k mutually independent Markov random walks its s.d. becomes the k-fold convolution πk* on πk* fulfillsv∈Vfv = k. In this case, f can be considered as traffic configuration where fv counts the vehicles at vertex v. In fact, πk* corresponds to the multinomial distribution with parameters k and π, see Chapter 35 in [k and πv for all v ∈ V, respectively. Note that the process v in the Markov traffic.The s.d. ge 75 in . Since ter 35 in . MoreoveP is the uniform Markov kernel on G then we obtain the standard random walk of the graph theory, see the survey [A similar model to Markov random walk is proposed in  where bie survey .k = 1 in the sense that it describes the movement of a random vehicle which follows the stochastic rules defined by the Markov kernel. On the other hand, the Markov traffic provides a mathematical model for describing the traffic of k vehicles on a road network. Note that the independence assumption seems reasonable and not too strong because the vehicle controls are working separately form each other. For a pair u, v ∈ V the notation u ⇒ v will mean that  ∈ E ∪ S, i.e., either u → v or u = v. If X1, X2 are random functions on V then X1 ⇒ X2 means that the two-dimensional distribution of  is concentrated on E ∪ S. Clearly, for any Markov random walk Xt ⇒ Xt+1 ⇒ … ⇒ Xt+n for all t and u to vertex v with probability puv. The second-order Markov random walk (traffic) on the line road network, where the vehicles move from edge to edge, can also be defined similarly, see [A Markov random walk is an individual Markov traffic with rly, see .Q = (quv) on V × V as quv ≔ πupuv, u, v ∈ V. Then, Q is a two-dimensional s.d. on G in the following sense:Using the concept of two-dimensional s.d. a Markov traffic can be reparametrized in the following way. Introduce the two-dimensional distribution Definition 3. A matrix Q = (quv)u,v∈V is called two-dimensional stationary distribution on G if (i) quv ≥ 0 for all u, v ∈ V and quv = 0 for all u, v ∈ V such that  ∉ E ∪ S ; (ii) ∑u,v∈Vquv = 1 ; and (iii) ∑v∈Vquv = ∑v∈Vqvu for all u ∈ V .Q on G is called (strictly) positive if quv > 0 for all u, v ∈ V such that (u ⇒ v) u → v.A two-dimensional s.d. G coincide with each other. Clearly, for a Markov traffic, Q defined above is a positive two-dimensional distribution on G. Q can also be considered as a p.d. on the state space E ∪ S, i.e., if we extend the set V′ of vertices of L(G) as V′ = E ∪ S, on the line digraph. Thus, Q can be interpreted as the distribution of the vehicles on the edges of the road network, i.e., on the line digraph, see formula (11) in [Q can also be visualized on the edges, see, Xt, Xt+1), Q.Property (iii) states that the two (row-wise and column-wise) marginal distributions of a two-dimensional s.d. on  11) in . The dis1 in . ThG. One can easily see that Denote by P = (puv) defines a G-compatible Markov kernel with s.d. π = (πu) on G. Thus, a Markov traffic defined by the quadruple  can be introduced by an equivalent way through the triplet . However, it will turn out later that, from a statistical point of view, the parameter matrix Q can be estimated in a computationally more efficient way than the pair of transition matrix P and its s.d. π.Conversely, for a positive The simulation method proposed in this paper is based on the ergodicity of Markov traffic which follows from the ergodic theory of finite Markov chains.π0 be an initial distribution on V and define the n-th absolute p.d. πn by the recursion P and Q on V is defined as (PQ)uw ≔ ∑v∈Vpuvqvw, u, w ∈ V. If G is strongly connected and we start from the unique s.d. π0 = π of P, see Theorem 1, then πn → π as n → ∞. However, in general, the sequence π0 even if the s.d. is unique. However, if we consider the average of the n-th absolute p.d.’s in time, we have the convergence to the unique s.d. The Markov kernel which satisfies this property is called ergodic.Let G is strongly connected then any G-compatible Markov kernel P is ergodic and the average Markov kernel An converges, i.e.,n → ∞, where π is the unique s.d. of P. Moreover, the limiting probabilities of the time averages of the absolute p.d.’s satisfyn → ∞ for all initial p.d. π0.The following result is based on Theorem 4.1 in Chapter 3 of . If a rof(π) ≔ ∑v∈Vfvπv of p.d. π, where n → ∞, see Theorem 4.1 of [In applications, along absolute p.d.’s, we may also be interested in some functionals of these distributions, e.g., the number of vehicles in a region of the road network. Define the functional F. Then,  can be em 4.1 of .n-th absolute p.d.’s we need the additional assumption of aperodicity for G, see Theorem 2.1 in Chapter 4 of [G is an aperiodic, strongly connected road network and P is a G-compatible Markov kernel on it, then the sequence of Markov kernels Pn, Π. Moreover, the limit of the sequence of n-th absolute p.d. πn is the unique s.d. π to the Markov kernel P which is independent of the initial p.d. π0. For any functional F we also have that F(πn) converges to F(π) as n → ∞ on an aperiodic, strongly connected road network.Instead of time averages, in order to achieve the convergence of ter 4 of . If G isG-compatible Markov kernel is derived in the same way. Let n-th absolute p.d., respectively, for the ith vehicle, i = 1, …, k. Then, the ergodic property of Markov traffic, similarly to .The ergodicity of Markov traffic with any larly to , can be in)(f)→πvas n → ∞G-compatible Markov kernel can be approximated and thus explored by long run behavior of the traffic flow on the road network. A visualization of the convergence of Markov traffic simulation to its s.d. is presented in Fig 10.The convergence results  and 7)  7) guaraContains complete trajectories, i.e., the availability of only the start and end points is not sufficient, intermediate trajectory points must also be available.The trajectory points must be sampled at a high enough frequency, so that the distance between consecutive points should not be too large, .The dataset is sufficiently large. It should cover a long enough period of time, preferably uniformly. The number of trajectories per day should be of the order of thousands.Trajectories should cover a relatively small geographic area, e.g., a city or a district.Vehicles should not follow a fixed route, e.g., public transport bus trajectories are not suitable.Publicly available for research purposes.For our experiments, we needed a dataset of real-life traffic trajectory data. In our terminology, a trajectory is a sequence of data that provides information about the path of a vehicle moving from a start to an end point, associating geographic coordinates with timestamps. We required a dataset that satisfies the following criteria:These requirements were satisfied by the Taxi Trajectory Prediction (TTP) dataset from Kaggle. The dataset covers a period of one year from July 1, 2013 to June 30, 2014. It is split into a training and a test set, the former contains 1,710,670 trajectories, and the latter contains 320. The trajectories were collected in the city of Porto, Portugal, with a sampling rate of 15 seconds. First, we created a subset of the dataset, filtered to coordinates between W8.6518, W8.5771, N41.1129, N41.1756, see http://wiki.openstreetmap.org/wiki/API. OSM uses two formats for exporting map data, namely OSM XML and PBF. Software libraries for parsing and working with OSM data are available for several programming languages, see https://wiki.openstreetmap.org/wiki/Frameworks.OpenStreetMap (OSM) is a community project to build a free map of the world to which anyone can contribute. Data is available under the Open Data Commons Open Database License (ODbL). The representation and storing of map data is based on a simple but powerful model, that uses only three modeling primitives, namely, nodes, ways, and relations: 1. A node represents a geographical entity with GPS coordinates. 2. A way is an ordered list of at least two nodes. 3. A relation is an ordered list of nodes, ways, and/or relations. All of these modeling elements can have associated key-value pairs called tags that describe and refine the meaning of the element to which they belong. Users can export map data at the OSM web site manually, selecting a rectangular region of the map. Alternatively, map data can be extracted via web services, see We started our processing by building a graph from the OSM map of Porto, with the same bounding box as the filtered dataset. Specific nodes of the OSM file become the nodes in the graph. Because we only need those nodes that can be reached via vehicles, we had to filter the OSM file and collect only specific types of way nodes. In the OSM file, a way is a sequence of OSM nodes, so naturally, the nodes of ways become nodes in the graph. For every node we store the node’s OSM ID, and its coordinates. We also insert an edge into the graph between every nodes in way. The weight of an edge is given by the squared distance between the nodes, which we calculate from the OSM file’s data. We used pyosmium library for processing the OSM files and the NetworkX Python library for building the graph.After building the graph we process the list of trajectories. Because the trajectories are given in GPS coordinates, we first have to translate those coordinates into OSM node IDs. For every coordinate in a trajectory, we search for the closest way node’s coordinates in the built graph, so the result nodes have the same domain as the built graph’s nodes. Obviously, the original trajectories made up of GPS coordinates does not have the same scaling as the OSM map. The coordinates in the trajectory are sampled in regular, but larger time intervals than the OSM, so they are not aligned. In order to match a trajectory to a way in our graph, we had to perform an interpolation on the result list of node IDs, so we ran a Dijkstra’s shortest path algorithm on our graph between every node IDs for every trajectory. Because the OSM database contains errors, it can happen that in real life a route exists between two given places, but in the OSM database, there are no existing routes between those nodes that are representing the given places. In this case, we cut the faulty trajectories into pieces. The result of this process is an aperiodic strongly connected road network augmented by the ideal vertex 0, with a set of trajectories on the road network.G, P, π, k) or the triplet  using observed data. To estimate G we have to explore the road system under study by identifying the set V of vertices and the set E of directed road segments. Fortunately, this exploring has already been done by a few organizations, see, e.g., the Google Maps and the OpenStreetMap. However, we should note that, in case of GPS-based trajectory data, we have to fit the data to the applied map system which is not an evident task at all. In the present paper, we propose a method for estimating the two-dimensional stationary distribution Q immediately instead of the pair  of a transition matrix and its stationary distribution using mobile sensor data which may be gathered by vehicles, passengers etc. In this case, we have trajectories data which consists of the sequences of consecutive vertices, like in the TTP dataset. By   pairs in the i-th trajectory. One can see that since {Xi} is a proper Markov random walk we have u, v)∉E ∪ S. Thus, the support of the two-dimensional frequency matrices N ≔ (nuv)u,v∈V, i = 1, …, k, is a subset of E ∪ S, i.e., they are weakly G-subordinated matrices. Clearly, n − k is the corrected sample size. Introducev ∈ V, i.e., sv denotes the number of trajectories which start at vertex v and ev denotes the number of trajectories which terminate at vertex v, respectively. Denote the one-dimensional marginal frequencies of N by nv+ ≔ ∑u∈Vnvu and nv+ ≔ ∑u∈Vnuv, v ∈ V. We obtain thatv ∈ V, where nv denotes the number of vertex v in all trajectories. Finally,s and e on V as s ≔ (sv)v∈V and e ≔ (ev)v∈V, respectively. Then, (1⊤(e − s) = 0, i.e., the vectors e − s and 1 are orthogonal.Suppose that, for a Markov traffic, we observed a random sample of trajectories {∑v∈Vev=k.Define tP is given by the maximization of the conditional loglikelihoodpuv, u, v ∈ V such that u ⇒ v, under the constraints pu+ = 1 for all u ∈ V. A solution of this constrained optimization problem is u, v ∈ V if nu+ > 0 and nu+ = 0 where δ denotes the Kronecker delta. The maximum likelihood estimator π is derived by the solution of the global balance equation π. Thus, the maximum likelihood estimator Q is given by u, v ∈ V. In the sequel, a direct method is proposed for estimating the two-dimensional stationary distribution Q.The traditional maximum likelihood (ML) estimator Q based on the two-dimensional consecutive empirical frequency matrix N is V, satisfies the properties (i) and (ii) of Definition 3. However, the problem with this naîve estimator is that its row and column marginals are not necessarily equal, i.e., in general, it does not satisfy the asumption (iii) of Definition 3. Hence, we have to introduce a new estimator A naîve estimator for the two-dimensional stationary distribution G. Let A = (auv)u,v∈V and B = (buv)u,v∈V such that auv = buv = 0 for all u, v ∈ V where A and B be weakly G-subordinated matrices. The distance between A and B is defined asG is the Frobenius norm of the matrices of dimension |V| × |V| which vanish on the entries outside of E ∪ S.The optimality of the proposed estimator is defined by means of the least squares distance between matrices over M = (muv) denote a non-negative parameter matrix on G which satisfies assumptions (i) and (iii) of Definition 3, i.e., M is weakly G-subordinated and ∑v∈Vmuv = ∑v∈Vmvu for all u ∈ V. Then, one can easily derive a two-dimensional stationary distribution Q from M by its normalization defining as Q ≔ (1⊤M1)−1M.To formulate the objective function for estimating the two-dimensional stationary distribution it is convenient to weaken the assumptions of Definition 3 by leaving the normalizing assumption (ii). In the sequel, let k number of trajectories, using the Frobenius norm, the optimality criterion is defined as the weighted sum of squared errors (SSE):M is a non-negative parameter matrix satisfying assumptions (i) and (iii) of Definition 3, w = (wi)i=1,…,k are non-negative unknown weights, with N ≔ (Ni)i=1,…,k denotes the data, where Ni is the two-dimensional consecutive empirical frequency matrix for the ith trajectory, see (M and w deriving the weighted least squares (WLS) estimators Q is defined as neff gives the equivalent sample size related to the independent case which may be thought of as the information content of the observed data. Note that neff is not necessarily an integer and is different from n and n − k. Finally, wi gives the importance of the ith trajectory in the sample. One can see that longer trajectory implies higher weight.Based on ory, see . The staL of a digraph G is defined asA denotes the adjacency matrix of G and D ≔ diag{d+ + d−} is a diagonal matrix. Note that for the road graph G, since there is no loop, we have lvv = dvv = deg+(v) + deg−(v) for all v ∈ V. The main theorem of this paper is the following.To formulate our result on WLS estimation of Markov traffic we need some basic facts on the spectral theory of directed graphs, see  for detaTheorem 2. There is a unique pair i = 1, …, k, andLλ = s − e which satisfies the constraint 1⊤λ = 0  and ∘ denotes the entrywise (Hadamard) product of matrices.fined in . These Wneff depends only on the graph structure of the road network, which is independent of the data, the traffic direction vector s − e, and the corrected sample size. However, it does not depend on the data which are inside the trajectories.Based on the previous theorem, by , the effith trajectory, i = 1, …, k. The parameters of the objective function MSE are the effective sample sizes Q. The heuristic explanation of the need to use weights in formulas  likelihood estimators for Markov chains, see . The comformulas  and 14)MSE≔neff−ni, i = 1, …, k} are bounded, where k is large or tends to infinity. In our example, for the TTP dataset, the number of trajectories is above 80K with the mean length 40 and maximum length 2K, see s − e while The estimation theory of finite Markov chains goes back for a long time, see . In the The fundamental statement of Theorem 2, as one of the main result of this paper, is that the estimator λ in Theorem 2. This can be performed in a numerically effective manner by the eigenvalues-eigenvectors decomposition of the symmetric unnormalized  graph Laplacian matrix, see Proposition 1 and 2 in [The computationally intensive part of WLS estimator is the numerical solution of sparse linear equation system given for the Lagrange vector and 2 in . Remark and 2 in ), a lineQ as a parameter is defined by https://pypi.org/project/PyDTMC/). The codes and datasets of our simulation are available upon request.To evaluate the performance of the proposed WLS estimation method by comparing it to the traditional ML one discussed above, a simple simulation study was conducted at different sample sizes for small and medium road network. In the simulations, in order to mimic the real traffic, we tried to keep the length of trajectories low and the number of trajectories high compared to the size of the road network, similarly to the Porto example. The absolute bias of an investigated estimator https://github.com/rbesenczi/Crowd-sourced-Traffic-Simulator/blob/master/model-sources/Markovkernel/example_graph.py). The simulation parameters were k = 100, 200, 500, and 1000 number of trajectories with n = 3, 5, and 10 length. The absolute bias and its standard error do not depend on the length n and they are decreasing as k is increasing for both estimation methods. The latter is an expected result. Moreover, while for relatively small k the performance of the WLS and ML methods are similar, in the case of relatively large number of trajectories the ML estimator outperforms the WLS one a little bit. This phenomenon could be due to the asymptotic optimality of the ML estimator because the parameter k is enough large (1000) compared to the size of the road graph (5).k = 1000, 3000, and 5000 with n = 3, 5, and 10. In this scenario, the absolute bias and its standard error depend also only on k and are independent of the length n. However, there are significant differences between the performances of the two estimation methods (ML and WLS) related to the parameter k. On the one hand, the absolute bias of ML estimator is decreasing as k is increasing while it is constant for WLS estimator. On the other hand, the WLS estimator is better than the ML one in case of k = 1000 but worse in case of k = 5000. Since the former parameter setting is closer to the real traffic, this simulation corroborates the superiority of WLS method based on two-dimensional s.d. against the traditional maximum likelihood. Finally, in this scenario, the scale of the SE’s indicates that the WLS estimator is more stable than the ML one See In the second simulation scenario, a strongly connected subgraph, which contains 1000 vertices, of Porto’s road network was chosen . The entries of Markov kernel were generated randomly. The simulation parameters were We have also implemented the model in the OOCWC system in order to apply our simulation method for real large-scale problems. First, we have filtered the TTP dataset. Then, we have created the Markov kernel from the filtered dataset, so all nodes of the simulation graph will have the corresponding transition probability vector. We should note, however, that not all nodes can be found in the Markov kernel, because it can happen that the dataset does not completely cover the whole map, i.e., not all nodes are part of a trajectory. In this case, we set uniform distribution for the corresponding node. Finally, we had to modify the basic operation of the simulation algorithm. In the original implementation, the cars are moving on the map quite randomly. Now, a car selects the next node based on the transition probability vector of the current node. For this, we use the pseudo-random number generation engine from the Boost Random library that is based on the method presented in paper .Let’s consider an example. We are at the graph vertex (or intersection) of OSM node ID 1110673569 . The total transitions of this node  in the dataset is 1,649. The transitions to the neighbor nodes are shown in k = 5, 000, 10, 000, 20, 000, 30, 000 and 50, 000 units. The simulation starts when all simulation units are added to the map. The initialization phase of the simulation adds traffic units to the map. Each unit is placed to an OSM node, i.e., on a vertex of the simulation graph. There exist two ways to do this, one is following a prescribed distribution (e.g. uniform), the other is following measured data. In our test case, we initialized simulations with fictional measured data. We put units only to the streets Rua de Antero de Quental, Rua da Constituição and Rua da Boavista , i.e., the simulation starts from the traffic configuration which is concentrated on three nodes of the road graph. In addition, we can set the number of the simulation units. We run simulations with The RCE produces a logfile that contains the position of every simulation unit in every simulation step. From this file, we calculate the number of cars by streets in every minute, so we can observe the change of distribution of the cars. In addition, we calculated the s.d. of cars for streets in the city of Porto, see To obtain a quantitative measure that describes the “goodness” of our simulation algorithm, we applied the Pearson’s chi-squared test. We expect, by the ergodicity of the Markov traffic, that during the simulation, independently from the initial distribution, within a certain time period, the distribution of the cars become close to the previously calculated stationary distribution. Finally, we should note some implementation details and possible drawbacks of this model that may have an impact on the model’s overall performance.π, namely, negative values may present in the results. We need to handle this problem when we calculate the Pearson’s chi-squared test. We chose to shift every value of π until we get a sum of 1 for π.In small and medium graphs, the proposed algorithm and its implementation performs as it is expected. But in the case of our Porto example, where the graph has 33,961 nodes and 53,126 edges and the TP matrix is very sparse, numerical problems may occur. One problem can occur when we calculate the Some minor issues can occur with the map database and the differences between the Porto dataset and the OSM data. In some cases, we could not calculate a route between two consecutive trajectory points using OSM data. This can happen because of the imperfection of the OSM data or false GPS measurement. We handle this case by splitting the trajectory into pieces.π for a subgraph of the whole map. During the simulation the units can traverse the whole map graph, so, it can happen that a traffic unit reaches an edge which is not part of the subgraph where we know the s.d. π. During the calculation of the Pearson’s chi-squared test, we consider only those cars that are present on the road network, where the s.d. π is known.Another minor issue arises in the calculation of the Pearson’s chi-squared test. Since the OSM Porto map and the trajectory dataset do not cover each other perfectly, we only know the s.d. In this paper, we have described our traffic simulation model that is called “Markov traffic” based on tools from graph theory and Markov modeling. The aim was to provide a simulation method that is able to keep the distribution of the cars on the map in a steady-state on a large scale road network. We have proven that, under general assumptions, the stationary distribution (s.d.) is unique for any Markov transition mechanism on a wide class of road networks. An explicit formula has also been derived for the s.d. and the ergodicity of Markov traffic has also been proved.We have shown that the s.d., with the related transition mechanism, can be explored from observed data based on sample trajectories. We have provided a statistical method and proved its optimality by simulation with which we can create the Markov kernel necessary to obtain a Markov traffic on a given road graph. Using this kernel, we can initiate traffic simulations that provide a s.d. of the cars on the map.To provide an example for creating this kernel file, we have used a publicly available dataset, namely the Taxi Trajectory Prediction dataset. Our simulation uses OpenStreetMap, a free map database.https://github.com/rbesenczi/Crowd-sourced-Traffic-Simulator/blob/master/justine/install.txt). Some simulation video is available at the YouTube channel of the first author at http://bit.ly/2FRpPxL.To test our theories, we have implemented the proposed model in our simulation program (RCE). We have run simulations and it has been proved to provide a s.d. on the map graph of Porto, Portugal. The whole project (including the RCE) is available for download Click here for additional data file.


Scientific Reports 10.1038/s41598-020-76726-7, published online 12 November 2020Correction to: The original version of this Article contained an error in the title of the paper, where the word “biofunctionalized” was incorrectly given as “biofunctionalaized”. This has now been corrected in the PDF and HTML versions of the Article.


ENORA and NSGA-II have been adapted to optimize the performance of the classifier based on three different machine learning metrics: accuracy, area under the ROC curve, and root mean square error. We have extensively compared the generated classifiers using our proposed method with classifiers generated using classical methods such as PART, JRip, OneR and ZeroR. The experiments have been conducted in full training mode, in 10-fold cross-validation mode, and in train/test splitting mode. To make results reproducible, we have used the well-known and publicly available datasets Breast Cancer, Monk’s Problem 2, Tic-Tac-Toe-Endgame, Car, kr-vs-kp and Nursery. After performing an exhaustive statistical test on our results, we conclude that the proposed method is able to generate highly accurate and easy to interpret classification models.The ease of interpretation of a classification model is essential for the task of validating it. Sometimes it is required to clearly explain the classification process of a model’s predictions. Models which are inherently easier to interpret can be effortlessly related to the context of the problem, and their predictions can be, if necessary, ethically and legally evaluated. In this paper, we propose a novel method to generate rule-based classifiers from categorical data that can be readily interpreted. Classifiers are generated using a multi-objective optimization approach focusing on two main objectives: maximizing the performance of the learned classifier and minimizing its number of rules. The multi-objective evolutionary algorithms  Supervised Learning is the branch of Machine Learning (ML) [Artificial Neural Networks (ANN) [Deep Learning Neural Networks (DLNN) [Support Vector Machines (SVM) [Instance-based Learning (IBL) [DLNN, for example, is a large mesh of ordered nodes arranged in a hierarchical manner and composed of a huge number of variables. DLNNs are capable of modeling very complex behaviors, but it is extremely difficult to understand the logic behind their predictions, and similar considerations can be drawn for SVNs and IBLs, although the underlying principles are different. These models are known as black-box methods. While there are applications in which knowing the ratio behind a prediction is not necessarily relevant, , there are other situations where the interpretability of a model plays a key role.ing (ML)  focused ing (ML) . The besks (ANN) , and, mos (DLNN) , as welles (SVM) , and Insng (IBL)  are archinterpretability of classification systems refers to the ability they have to explain their behavior in a way that is easily understandable by a user [accuracy, area under the ROC curve, and root mean square error), there is no universally accepted metric to measure the interpretability of the models. Nor is there an ideal balance between the interpretability and performance of classification systems but this depends on the specific application domain. However, the rule of thumb says that the simpler a classification system is, the easier it is to interpret. Rule-based Classifiers (RBC) [RBC as the number of its rules or as the number of axioms that the rules have. These metrics tend to reward models with fewer rules as simple as possible [RBCs are classification learning systems that achieve a high level of interpretability because they are based on a human-like logic. Rules follow a very simple schema:RBCs are so natural in some applications that they are used to interpret other classification models such as Decision Trees (DT) [RBCs constitute the basis of more complex classification systems based on fuzzy logic [LogitBoost or AdaBoost [The y a user . In others (RBC) ,9 are ampossible ,11. In gees (DT) . RBCs cozy logic  such as AdaBoost .multi-objective optimization problem. We define a solution as a set of rules , and establish two objectives to be maximized: interpretability and accuracy. We decided to solve this problem by applying multi-objective evolutionary algorithms (MOEA) [NSGA-II [ENORA [NSGA-II is very well-known and has the advantage of being available in many implementations, while ENORA generally has a higher performance. In the current literature, MOEAs are mainly used for learning RBCs based on fuzzy logic [Fuzzy RBCs are designed for numerical data, from which fuzzy sets are constructed and represented by linguistic labels. In this paper, on the contrary, we are interested in RBCs for categorical data, for which a novel approach is necessary.Our approach investigates the conflict between accuracy and interpretability as a s (MOEA) ,16 as me[NSGA-II  and ENORI [ENORA . They arI [ENORA ,19,20. Nzy logic ,24,25,26ENORA and NSGA-II, and the well-known rule-based classifier learning systems PART, JRip, OneR and ZeroR. In RBC for categorical data, and we propose the use of multi-objective optimization for the task of learning a classifier. In Breast Cancer, Monk’s Problem 2, Tic-Tac-Toe-Endgame, Car, kr-vs-kp and Nursery. The experiments allow a comparison among the performance of the classifiers learned by our technique against those of classifiers learned by PART, JRip, OneR and ZeroR, as well as a comparison between ENORA and NSGA-II for the purposes of this task. In This paper is organized as follows. In optimization [Mathematical programming [multi-objective constrained optimization problems [l objectives and m constraints, as follows:objectives) and combinatorial, and those with continuous variables. In combinatorial problems, we are looking for objects from a finite, or countably infinite, set The term mization  refers tgramming  deals wiproblems , which c follows:Min./Max.non-dominated set . A solution non-dominated if there is no other solution dominatesi . When a problem . A non-lgorithms  are typiMOEA ENORA [NSGA-II [binary tournament selection (Algorithm 2) and a rank function based on Pareto fronts and crowding (Algorithms 3 and 4). The difference between NSGA-II and ENORA is how the calculation of the ranking of the individuals in the population is performed. In ENORA, each individual belongs to a slot , therefore, the time complexity of ENORA is The EA ENORA  and NSGA[NSGA-II  use a μ+ished in ) of the  sorting . It comp O(l·N2) . HoweverAlgorithm 1Require:Require: 1:P with N individualsInitialize  2:PEvaluate all individuals of  3: 4:whiledo 5:  6:  7:whiledo  8:Parent1← Binary tournament selection from P   9:Parent2← Binary tournament selection from P  10:Child1, Child2←Crossover  11:Offspring1←Mutation(Child1)  12:Offspring2←Mutation(Child2)  13:Offspring1  Evaluate 14:Offspring2  Evaluate 15:  16:  17:end while 18: 19:N best individuals from R according to the rank-crowding function in population R 20: 21:end while22:return Non-dominated individuals from PAlgorithm 2 Binary tournament selection.Require:P {Population} 1:P 2:P 3:ifI is better than J according to the rank-crowding function in population Pthen 4:returnI  5:else 6:returnJ  7:end ifAlgorithm 3 Rank-crowding function.Require:P {Population}Require: 1:ifthen 2:return  3:end if 4:ifthen 5:return  6:end if 7:returnENORA and NSGA-II behave differently is as follows. NSGA-II never selects the individual dominated by the other in the binary tournament, while, in ENORA, the individual dominated by the other may be the winner of the tournament. B and C are selected for a binary tournament with NSGA-II, individual B beats C because B dominates C. Conversely, individual C beats B with ENORA because individual C has a better rank in his slot than individual B. In this way, ENORA allows the individuals in each slot to evolve towards the Pareto front encouraging diversity. Even though in ENORA the individuals of each slot may not be the best of the total individuals, this approach generates a better hypervolume than that of NSGA-II throughout the evolution process.The main reason ENORA is our MOEA, on which we are intensively working over the last decade. We have applied ENORA to constrained real-parameter optimization [NSGA-II algorithm was designed by Deb et al. and has been proved to be a very powerful and fast algorithm in multi-objective optimization contexts of all kinds. Most researchers in multi-objective evolutionary computation use NSGA-II as a baseline to compare the performance of their own algorithms. Although NSGA-II was developed in 2002 and remains a state-of-the-art algorithm, it is still a challenge to improve on it. There is a recently updated improved version for many-objective optimization problems called NSGA-III [mization , fuzzy omization , fuzzy cmization , featuremization  and featmization . In thisNSGA-III .Algorithm 4 Crowding_distance function.Require:P {Population}Require:P {Population}Require:I {Individual}Require:l {Number of objectives} 1:forldo 2:  3:  4:jth objective for the individual higher adjacent in the jth objective to the individual I  5:jth objective for the individual lower adjacent in the jth objective to the individual I  6:end for 7:forldo 8:ifthen  9:return ∞  10:end if 11:end for12:13:forldo14: 15:end for16:returnPART  is a wiT Method . Experimstrategy  typical lgorithm ), and thJRip is a fast and optimized implementation in Weka of the famous RIPPER  algorithm [RIPPER was proposed in [IREP) rule learner developed in [IREP and RIPPER work in a similar manner. They begin with a default rule and, using a training dataset, attempt to learn rules that predict exceptions to the default. Each rule learned is a conjunction of propositional literals. Each literal corresponds to a split of the data based on the value of a single feature. This family of algorithms, similar to decision trees, has the advantage of being easy to interpret, and experiments show that JRip is particularly efficient in large datasets. RIPPER and IREP use a strategy based on the separate-and-conquer method to generate an ordered set of rules that are extracted directly from the dataset. The classes are examined one by one, prioritizing those that have more elements. These algorithms are based on four basic steps  applied repetitively to each class until a stopping condition is met [Weka implementation, the procedure selects the condition with the highest information gain). In the pruning phase redundancy is eliminated and long rules are reduced. In the optimization phase, the rules generated in the previous steps are improved (if possible) by adding new attributes or by adding new rules. Finally, in the selection phase, the best rules are selected and the others discarded.lgorithm . RIPPER posed in  as a morloped in . IREP ann is met . These sOneR (One Rule) is a very simple, while reasonably accurate, classifier based on a frequency table. First, OneR generates a set of rules for each attribute of the dataset, and, then, it selects only one rule from that set—the one with the lowest error rate [ror rate . The setZeroR  is a clRBC specifically designed for categorical data. Then, we propose and describe a multi-objective optimization solution to obtain optimal categorical RBCs.In this section, we propose a general schema for an M rules, where each rule antecedent) takes values in a set consequent) takes values in maximum matching as reasoning method, where the compatibility degree of the rule Let association degree for the example c, that is:The classification (or output) of the classifier Therefore, the K instances with p categorical input attributes, j can take a category Let w is the number of classes of the output attribute and In the problem (Equation ), the fuon trees ,48. FiguENORA and NSGA-II have been adapted to solve the problem described in Equation .All individuals contain at least one rule for any output class between 1 and The initial population (Algorithm 5) is randomly generated with the following conditions:Algorithm 5 Initialize population.Require:Require:Require:Require:Require:Require:Require: 1: 2:forNdo 3:  4:ifthen  5:   6:else  7:Random   20:end if  21:end for 22: {Random integer values for adaptive variation}23:Random. The seself-adaptive crossover and mutation, which means that the selection of the operators is made by means of an adaptive technique. As we have explained , the prENORA and NSGA-II have been implemented with two crossover operators, rule crossover (Algorithm 9) and rule incremental crossover (Algorithm 10), and two mutation operators: rule incremental mutation (Algorithm 11) and integer mutation (Algorithm 12). Rule crossover randomly exchanges two rules selected from the parents, and rule incremental crossover adds to each parent a rule randomly selected from the other parent if its number of rules is less than the maximum number of rules. On the other hand, rule incremental mutation adds a new rule to the individual if the number of rules of the individual is less than the maximum number of rules, while integer mutation carries out a uniform mutation of a random antecedent belonging to a randomly selected rule.Both Algorithm 6 Variation.Require: 1: 2: 3:Self-adaptive crossover  4:Self-adaptive mutation  5:Self-adaptive mutation  6:returnAlgorithm 7 Self-adaptive crossover.Require:I, J {Individuals for crossing}Require:Require: 1:if a random Bernoulli variable with probability then 2:Random 3:Random, and have been used as a comparison for many algorithms on several occasions. In particular, in [AQ17-DCI, AQ17-HCI, AQ17-FCLS, AQ14-NT, AQ15-GA, Assistant Professional, mFOIL, ID5R, IDL, ID5R-hat, TDIDT, ID3, AQR, CN2, WEB CLASS, ECOBWEB, PRISM, Backpropagation, and Cascade Correlation. For our research, we have used the Monk’s Problem 2, which contains six categorical input attributes and a binary output attribute, summarized in Monk’s Problem 2 is the binary outcome of the logical formula:In July 1991, the monks of ular, in , they haaccuracy:K is the number of instances of the dataset i in ith instance in area under the ROC curve:ROC curve obtained with the classifier ROC (Receiver Operating Characteristic) curve [sensitivity versus the specificity index for a classifier varying the discrimination threshold value. Such a curve can be used to generate statistics that summarize the performance of a classifier, and it has been shown in [ROC curve can be computed as follows [sensitivity) is the proportion of positive instances classified as positive by the classifier specificity) is the proportion of negative instances classified as negative by t is the discrimination threshold. Finally, our third constrained optimization model is based on the root mean square error (RMSE):mean square error obtained with a classifier ith instance for the classifier ROC curve, and root mean square error are all well-accepted measures used to evaluate the performance of a classifier. Therefore, it is natural to use such measures as fitting functions. In this way, we can establish which one behaves better in the optimization phase, and we can compare the results with those in the literature.We have conducted different experiments with different optimization models to calculate the overall performance of our proposed technique and to see the effect of optimizing different objectives for the same problem. First, we have designed a multi-objective constrained optimization model based on the nstances  obtainedc) curve  is a grashown in  to be a  follows :AUCD(Γ)=ENORA and NSGA-II as metaheuristics in this particular optimization task, we use the hypervolume metric [error ratio, the generational distance, the maximum Pareto-optimal front error, the spread, the maximum spread, or the chi-square-like deviation, is that it can be computed without an optimal population, which is not always known [P, and is formulated as:P, and i. Subsequently, the hypervolume ratio (HVR) is defined as the ratio of the volume of the non-dominated search space over the volume of the entire search space, and is formulated as follows:HVR requires reference points that identify the maximum and minimum values for each objective. For RBC optimization, as proposed in this work, the following minimum :FDlo:FDloENORENORA, and three driven by NSGA-II), over both datasets, has been designed for the purpose of showing the aspect of the final Pareto front, and compare the hypervolume ratio of the models. The results of this single execution, with population size equal to 50 and 20,000 generations , are shown in A first single execution of all six models , we have executed again the six models in full training mode.To perform an initial comparison between the performance of the classifiers obtained with the proposed method and the ones obtained with classical methods (Breast Cancer dataset (this case), while, for the Monk’s Problem 2, it was set to 9. Observe that, since Breast Cancer dataset. In fact, after the preliminary experiments were run, it turned out that the classical classifier learning systems tend to return very small, although not very precise, set of rules on Breast Cancer, and that justifies our choice. On the other hand, executing the classical rule learners on Monk’s Problem 2 returns more diverse sets of rules, which justifies choosing a higher Monk’s Problem 2, that individual has seven rules. The comparison is shown in number of rules, percent correct, true positive rate, false positive rate, precision, recall, F-measure, Matthews correlation coefficient, area under the ROC curve, area under precision-recall curve, and root mean square error. As for the Breast Cancer dataset (observe that the best result emerged from the proposed method), in the optimization model driven by NSGA-II, with root mean square error as the first objective , excepting the tive see , only PArror see . The expcross-validated experiment for the Breast Cancer dataset, in which we iterated three times a 10-fold cross-validation learning process [percent correct, area under the ROC curve, and serialized model size of all results. Second, we designed a train/test percentage split experiment for the Monk’s Problem 2 dataset, in which we iterated ten times a 66% (training) versus 33% (testing) split and considered, again, the average result of the same metrics. Finally, we performed a statistical test over on results, to understand if they show any statistically significant difference. An execution of our methodology, and of standard classical learners, has been performed to obtain the models to be tested precisely under the same conditions of the experiment To test the capabilities of our methodology in a more significant way, we proceeded as follows. First, we designed a  process  and conspercent correct, area under the ROC curve and serialized model size. We proceeded as follows. First, we checked normality and sphericity of each sample by means of the Shapiro–Wilk normality test. Then, if normality and sphericity conditions were met, we applied one way repeated measures ANOVA; otherwise, we applied the Friedman test. In the latter case, when statistically significant differences were detected, we applied the Nemenyi post-hoc test to locate where these differences were. Breast Cancer dataset for each of the three metrics, and Monk’s Problem 2 dataset.The statistical tests aim to verify if there are significant differences among the means of each metric: Monk’s problem 2 dataset and for the following four other datasets:Tic-Tac-Toe-Endgame dataset, with 9 input attributes, 958 instances, and binary class  (kr-vs-kp), with 36 input attributes, 3196 instances, and binary class .From the statistical tests  between precision and interpretability than the rest of the classifiers. For the rest of the analyzed datasets, the accuracy obtained using ENORA is substantially higher. For example, for the Tic-Tac-Toe-Endgame dataset, ENORA obtains a PART obtains JRip obtains Car, kr-vs-kp and Nursery, we want to comment that better success percentage can be obtained if the maximum number of evaluations is increased. However, better success percentages imply a greater number of rules, which is to the detriment of the interpretability of the models.We would like to highlight that both the Breast Cancer  and Monk’s Problem 2 , using two different evolutionary algorithms, namely ENORA and NSGA-II, and three different choices as a performance metric, i.e., accuracy, the area under the ROC curve, and the root mean square error. Additionally, we have shown the results of the evaluation in 10-fold cross-validation of the publicly available Tic-Tac-Toe-Endgame, Car, kr-vs-kp and Nursery datasets.In this paper, we have proposed a novel technique for categorical classifier learning. Our proposal is based on defining the problem of learning a classifier as a multi-objective optimization problem, and solving it by suitably adapting an evolutionary algorithm to this task; our two objectives are minimizing the number of rules (for a better interpretability of the classifier) and maximizing a metric of performance. Depending on the particular metric that is chosen, (slightly) different optimization models arise. We have tested our proposal, in a first instance, on two different publicly available datasets, PART and JRip, but only indirectly. Finally, the idea underlying our approach is that multiple classifiers are explored at the same time in the same execution, and this allows us to choose the best compromise between the performance and the interpretability of a classifier a posteriori.Our initial motivation was to design a classifier learning system that produces interpretable, yet accurate, classifiers: since interpretability is a direct function of the number of rules, we conclude that such an objective has been achieved. As an aside, observe that our approach allows the user to decide, beforehand, a maximum number of rules; this can also be done in embedded future selection mechanism. In fact, all attributes are  used in every rule of a classifier learned by our optimization model. By simply relaxing such a constraint, and by suitably re-defining the first objective in the optimization model , the resulting classifiers will naturally present rules that use more features as well as rules that use less . Although this approach does not follow the classical definition of feature selection mechanisms (in which a subset of features is selected that reduces the dataset over which a classifier is learned), it is natural to imagine that it may produce even more accurate classifiers, and more interpretable at the same time.As a future work, we envisage that our methodology can benefit from an multi-objective differential evolution (MODE) for rule-based classification for inclusion in the Weka Open Source Software issued under the GNU General Public License. The implementation of other algorithms, such as MOEA/D, their adaptation in the Weka development platform and subsequent analysis and comparison are planned for future work.Currently, we are implementing our own version of