To overcome these issues, this paper proposes a x and y position based on inductive saliency between two sets of three-phase coils. The proposed idea is applied on a combined winding self-bearing machine which does not require additional suspension force winding. No additional search coil placement for Self-bearing machines do not contain physical bearings but magnetic bearings. Both rotor rotary and spatial positions displacement are required in these types of machines to control the rotor position while it is levitating. Self-bearing machines often use external sensors for  The remal axes) ,3,4.Conventional self-bearing machines typically comprise two sets of independent windings in the stator for producing torque and suspension force, i.e., separated winding (a torque winding and a suspension winding). For compact size self-bearing machines, combined winding that uses the same coils to create both suspension forces and torque current have been developed ,6.Position sensors for closed-loop rotor spatial  change. This is due to the unbalanced air-gap length between phases [Phase shown in , where Ressed by , where Lmovement ,9. Note n phases .n is equal to 1, m is equal to 2, and vice versa. Clark transform, Six-phase voltage models can be presented as two sets of αβn-axes  and 5)..αβn-axes, Kc, in  is used.The HF voltage characteristic model in stationary reference frame is represented by  and 7),,7), obtation see .x- and y-spatial rotor position displacement. This dependency will be used for rotor It will be shown in Finally, since shown in  exist amThis section presents the inductance dependency in rotor spatial position displacement   αβ1 refeLβ1β1 in . Therefor is the per-unit air-gap length. x- and y-axes. The effect of In this subsection, inductance sensitivity analysis is performed. Air-gap length in each phase is modeled as in  using EuTaking the dominant components, per unit air-gap length is calculated in –20): (1: (120): x and y rotor position displacement, Equivalent air-gap area, tance to , taking tance to  is calculated in . The senx and y rotor position displacement from the simplified inductance model can be summarized as follows:x-axis rotor movement. For x-position self-sensing, main inductances are required.Main inductances, y-axis rotor movement. For y-position self-sensing, cross-coupling inductances are required.Cross-coupling inductances, Cross-coupling inductances, The main and cross-coupling inductances in the first and the second set of three-phase coils have opposite sensitivity to The conclusions of inductance variation in n, while the other set of three-phase coils current, m, are set to zero  is performed to verify the conclusions from the simplified reluctance model. A stacking factor of 0.95 is used for the stator with N35 grade NdFeB permanent magnet on the rotor. Self-bearing machine geometry and simulation conditions for FEA analysis are summarized in  to zero  and 26)Dos, Dis, to zero –30). Gi. GiDos, n = 1, m = 2, i = 1, j = 2. Note that the magnitude of the cross-coupling inductances between three-phase coil sets are much smaller (about 1/5 times) than the self inductances shown in The cross-coupling inductance between the first and the second set of three-phase coils, shown in ions see . Therefos is the differential operator. Park transform in  as in (Equations  and 7)  7) can bsform in  with ref αβ-axes  where θi2) as in  to use d will be . The res will be , where Ifined in .x and y rotor position? This may be explained referring to , a, ax and rring to . The mainator of , VHF2πfHshown in  and 38)x and y rx and y rotor position are estimated, from the HF current responses, , to position, e.g., millimeter (mm), given HF injection conditions, i.e., x-axis offset constant to calibrate the x-axis referencing position. The procedure to select the self-sensing gains is as follows:x = y = 0 mm. first, ^ = 0 in  and 40)kgx and ksecond, Exciting both αβ2 with  as in Fisponses,  and 40)αβ1 and αsponses,  and (40)x and y rotor positions are estimated using (ed using  and 40)xy-positix- and y-axes is 0.1 mm. The center position of the rotor with respect to the stator is position controlled by the linear motion stage, and used as an absolute position reference during the This section shows the experimental results of the x = 0 mm and y= −1 mm, are shown in x and y positions, The self-sensing experiment conditions are summarized in ^, using  and 40)vα1HF*, vy-positions while the x-positions are varied from −1 mm to 1 mm in incremental steps of 0.5 mm, see x-position variation while To further show viability of the y-position is varied while holding the x-position constant. y position as expected in (x- and y-position results are shown in ected in  and 38)y-positioRotor x- and y-rotor position, simultaneously utilizing both main- and cross-coupling inductance variation. The excitation frequency is carefully selected to minimize the interference to the closed-loop operation of the bearingless machine drive system. Demodulating the coupled HF current, The model for machine inductances’ sensitivities with respect to the


Men with light eyes lack the dominant gene allele that codes for dark-brown eyes. Pairing with a woman who lacks the same allele must increase paternity confidence in these men, because any children with dark eyes would be extremely unlikely to have been fathered by them. This notion implies that men with light (blue or green) eyes should (1) prefer light-eyed women, especially in a long-term context, and (2) feel more threatened by light-eyed than by dark-eyed rivals. Yet because choosiness is costly and paternity concerns are entirely driven by the prospect of paternal investment, any such inclinations would be adaptive only in men who expect to invest in their children. Here I test these ideas using the data of over 1000 men who rated the facial attractiveness of potential partners, and the threat of potential rivals, whose eye color had been manipulated. Light-eyed men liked light-eyed women better (particularly as long-term companions), and feared light-eyed rivals more, than did dark-eyed men. An exploratory analysis showed that these large, robust effects disappeared in men who had felt rejected by their fathers while growing up—suggesting that such men are not expecting to invest in their own children either. It is a wise father that knows his own child. The Merchant of Venice, Act 2 Scene 2— William Shakespeare,Males can never be certain they have fathered their mates’ offspring. What they can do is adjust parental care to the probability of being genetically related to them Trivers, . There iand that this ability is highly imperfect.In our species, too, paternal investment is strongly related to paternity confidence reliably as to the other  permits to predict with high accuracy whether someone’s eyes are brown or blue  genes than would light-eyed men with the opposite preference, or no preference. This leads to the prediction that, other things being equal, men with light eyes should be especially attracted to women with light eyes. This has been shown to be the case in a study with 44 male participants who were asked to rate, from photographs, the attractiveness of faces with blue or brown eyes  and either brown or dark brown (dark-eyed men: N = 805), for a total of 1237 individuals; of these, 621 reported currently having a partner. Data were collected in accordance with guidelines approved by the local Psychological Research Ethics Committee; all participants gave informed consent.The total sample included 1440 men . With very few exceptions, participants were Italian; they were mostly recruited via links posted on Italian universities’ online social networks and other social media, such as Facebook groups. Men who self-identified as heterosexual or bisexual with a preference for female partners were directed to the current study, whereas men who self-identified as homosexuals or bisexual with a preference for male partners were directed to a separate one. The analyses presented here include all participants whose eyes were either blue or green  term relationship, which of these two people would you prefer?” Possible responses were “the one on the left” and “the one on the right.” The other questions were, “For a long- (short-) term relationship, how attractive do you find the person on the left?” and “For a long- (short-) term relationship, how attractive do you find the person on the right?”, each followed by a 0–10 scale going from “not at all attractive” to “very attractive.” Long-term and short-term relationships were defined as follows: “By a long-term relationship, we mean someone you may consider leaving your current partner for, or with whom you may like to create a stable relationship, leading to cohabitation or marriage, in case of a breakup with a current partner or if you are single”; “By a short-term relationship, we mean a single date accepted on the spur of the moment, an affair within a long-term relationship, or a one-night stand.”At the end, participants completed a questionnaire which asked about their eye color and that of their parents and current partner, if they had one. . Each pair was followed by three questions. The first was, “If you suspected your partner was being unfaithful to you, which of these two men would you be more jealous of?” Possible responses were “the one on the left” and “the one on the right.” The second and third questions were, “How jealous would you be of the man on the left?” and “How jealous would you be of the man on the right?”, each followed by a 0–10 scale from “not at all jealous” to “very jealous.”Light- and dark-eyed participants were not significantly different in any of the demographic characteristics measured in the study, such as age, ethnicity, education level, sexual orientation, partnership status, and mean partnership duration. Stimuli  and procedures  that were counterbalanced between subjects turned out to have been presented to light- and dark-eyed participants with the same frequency.  were not Italian or raised but not born in Italy. The pattern of results remained the same if the data of these participants were discarded. Only results directly relevant to the predictions of the paternity-uncertainty hypothesis are reported here; the complete set of results can be found in the Supplementary Materials.Men’s preference for light-eyed women was computed as the number of choices of light-eyed female faces, relative to the total number of choices, in response to the question “If you were looking for a long- (short-) term relationship, which of these two people would you prefer?” This index could range from 0 (light-eyed face is never chosen) to 1 .F = 93.33, p < 0.0001. Light-eyed men preferred women with light eyes more than dark-eyed men did, F = 37.30, p < 0.0001, η2p = 0.03—an effect which is about the size of juries’ tendency to be more influenced by credible than noncredible witnesses  = 8.49, p = 0.004.A repeated-measures ANOVA was carried out on these preferences, with a within-subject factor of relationship context  and a between-subject factor of own eye color . Light eyes were liked better in short- than in long-term partners, p’s < 0.0001, one-sample t). This core preference for light eyes has also been found in women , choosing a light-eyed partner confers an extra selective advantage to a man if he is light-eyed but not if he is dark-eyed.Overall, light eyes looked more attractive than dark ones  to 1 .p’s < 0.0001, one-sample t). These choices were analyzed with a univariate ANOVA with a fixed factor of own eye color . Crucially, having light eyes increased the perceived threat of light-eyed rivals relative to dark-eyed ones, F = 16.62, p < 0.0001, η2p = 0.03.In a potential competitor, light eyes looked more ominous than dark ones  = 13.10, p = 0.0003.  = 0.87, p = 0.350.) As shown by separate ANOVAs, having light eyes significantly increased the preference for light-eyed partners in men who had felt accepted by their father, F = 52.97, p < 0.0001, η2p = 0.07, but not in those who had felt rejected by him, F = 1.78, p = 0.183  = 7.69, p = 0.006; in participants who had felt rejected it was not, F = 1.49, p = 0.222. A similar effect emerged in the univariate ANOVA on partnered men’s jealousy of rivals: paternal rejection  interacted with own eye color , F = 4.09, p = 0.044. That is, having light eyes increased the relative threat of light-eyed rivals more in men who had felt accepted by their father than in those who had felt rejected by him  interacted with own eye color , F1, 106 = 13.1F1, 106 = 13.1p < 0.0001; jealousy: beta = 0.142, p = 0.007); and that the effect was dampened by paternal rejection .Results were similar if paternal rejection was left in its original, continuous form rather than split along the median; and if attraction and jealousy were computed not as proportion of choices, but as differences between the ratings given to light vs dark eyes on the 0–10 scale. Multiple regression analyses Norman,  showed t2p = 0.03 vs 0.008, that is Cohen’s d = 0.35 vs 0.18) or partialled out from one another. Second, as shown by separate ANOVAs, the effect of own light eyes was independently significant in men with light-eyed mothers, F = 13.75, p = 0.0002, and in men with dark-eyed mothers, F = 7.61, p = 0.006. Thus, the connection between having light eyes and liking light eyes shows up along with, not because of, sexual imprinting.Men are attracted to potential partners whose eye color resembles their own mother’s , hence it is sensible to ask whether light-eyed men’s preference for light-eyed women could be a mere byproduct of this inclination. Assortative mating might be direct, by eye color, or indirect—most plausibly by ethnicity, which could create the illusion of assortative mating by eye color just because eye color and ethnicity covary. However, even disregarding the detail that all participants had the same ethnicity, neither interpretation stands up to scrutiny. First, the effect of own eye color on preference is observed for light-eyed men but not for dark-eyed ones. In Fig. Here, I have put to a triple test the idea that light-eyed men should have evolved a particular preference for light-eyed women as potential partners and a particular concern with light-eyed men as potential rivals. I have done so with the help of the data of over one thousand men who rated the facial attractiveness of potential partners, and the threat of potential rivals, whose eye color had been manipulated. On this substantial sample, I found that light-eyed men see light-eyed women as more attractive than dark-eyed men do. The effect . Third, all these effects  ought to parallel men’s expectation to invest in their children. Hence, on the grounds that men who have experienced less affective investment from their father tend to repeat this pattern with their own children , relationship context , and potential rival’s eye color  were tested specifically by the study’s design. The role of paternal rejection was not predicted ahead of time, implying that this finding ought to be regarded as exploratory. Yet, that a man’s recollections of paternal rejection cancel, independently, all these effects suggests that such recollections do reflect a man’s disinclination to invest in his future children.Note importantly that, via its impact on the fitness of children, paternal investment has an impact on the fitness of mothers as well. Thus, the paternal-uncertainty hypothesis by no means predicts that women should be neutral with respect to eye-color preference (indeed, they are not: Bressan & Damian, Considering that proper mirrors were unavailable for most of our evolutionary history, one may wonder how people’s eye color could possibly inform their choices. One may think that the evolution of a preference for mates whose eye color looks like our own need not be more complex than that of a preference for people whose facial traits do—which has been demonstrated repeatedly and begins well before adulthood (Richter et al., It has been argued (Rantala & Marcinkowska,


We explored the results of fixed point, coincidence point and coupled coincidence point for the mappings in an ordered metric spaces. Our results generalized and extended the well-known results in the literature. Some numerical examples are provided for justifying the results obtained.b-metric space which satisfies a generalized week contraction condition. Furthermore, these results are extended for two self mappings for obtaining coincidence point, coupled coincidence point and coupled common fixed point in the same context. A few examples are presented to support the findings.Some fixed point results are found for a self mapping in a partially ordered  It has a wide range of uses in mathematical research and scientific applications. It was first established by Bakhtin in [b-metric space and then applied their results to quadratic integral equations. Common fixed point results for generalized weak contractions in the same context was studied by Aghajani et al. [b-metric space were discussed by Aleksić et al. [b-metric spaces was examined by Aydi et al. [b-metric space with necessary topological properties.A khtin in  and evenkhtin in . Later, khtin in , Allahyakhtin in  investigi et al. . Also, ti et al. . Some reć et al. . Common i et al. , Seshagii et al. , 24, 25 b-metric spaceIn this paper, we introduced the following generalized weak contraction condition which involve the altering distance functions y et al. , Bhaskarmentclasspt{minimay et al.  and Seshy et al. , 24, 25.mentclass2pt{minimy et al. , Huapingy et al. , Roshan y et al. , 19 and y et al. –23 for tIn the whole paper, we use the following nations for the altering distance functions: b-metric space.We start this section with the following fixed point result in a complete partially ordered Supposebe a complete partially ordered b-metric space withAssume that a continuous self mappingonis non-decreasing with respect toand satisfies the condition . If for somesuch thatthenhas a fixed point inmentclass2pt{minimThe proof is trivial for entclass1pt{minimaFurthermore the continuity of We have the following result in which the mapping According to Theorem 2.1, a non-continuous self mappinghas a fixed point ifmeets the condition (6):n, i.e., As from Theorem n, then  an an14) an1s) from . Accordi1s) from , we haveWe have the following result without the continuity property of Ifhas the property in Theorem 2.6thatthen the weakly compatible mappingshave a coincidence point. Besides that, whenandcommute at their coincidence points, thenhave a common fixed point inn. Next to show that As from Theorem }μ. From , we haveb-metric space Consider the partially ordered Letbe a complete partially ordered b-metric space. Assume that a mappingsatisfies the condition  and,are continuous,has mixed-monotone property and commutes withSuppose, if for some such thatandthenandhave a coupled coincidence point inondition  and,\doFrom Theorem 2.2 of , there w=ϖn+1 in , we get2,σ=ζn in , we get3Suppose that if hen from , we get and thus  holds. S1 thence  follows 1 thence .Now, we can deduce from  that \doa 3.1 of . TherefoSupposebe a complete partially orderedb-metric space. Let a continuous mappinghas a mixed monotone property and satisfies the contraction conditions below for anysuch thatandandi.ii.whereandIf there existssuch thatandthenhas a coupled fixed point inA unique coupled common fixed point forandexists in Theorem 2.9, if for everythere is somesuch thatis comparable toand toFrom Theorem Now, assume the followingfrom Eq. , we get4Assume Iforin Theorem2.11, thenandhave a unique common fixed point inAssume that mma 2 of , we haveThe result can also be seen in the case of The same conclusions can also be seen as in Theorems  Although uence of , the conmentclasspt{minimasults of  as well Now depending on the continuity of a metric Let b-metric space. Assume that For If If Define a metric b-metric space with Suppose Suppose that Let Since, b-metric space, the existence and uniqueness of a fixed point for a self mapping which satisfies a generalized weak contraction condition with two rational auxiliary functions are discussed. These results are further generalized for two self mappings in the same context and proved the existence of coincidence point, coupled coincidence point and coupled common fixed points. Also, shown that these results are generalized the well known existing results in the literature. Some numerical examples are given to justify the obtained results.b-metric space to acquire triple, quadruple fixed points.These results can be extended by involving more mappings in partially ordered These contractions can be used to obtain a coincidence point, coupled coincidence point and coupled common fixed points for the mappings in various ordered metric spaces with required topological properties like monotone non-decreasing, mixed monotone, compatible etc.In complete partially ordered


The high longitudinal and lateral coherence of synchrotron X-rays sources radically transformed radiography. Before them, the image contrast was almost only based on absorption. Coherent synchrotron sources transformed radiography into a multi-faceted tool that can extract information also from “phase” effects. Here, we report a very simple description of the new techniques, presenting them to potential new users without requiring a sophisticated background in advanced physics. We then illustrate the impact of such techniques with a number of examples. Finally, we present the international collaboration SYNAPSE (Synchrotrons for Neuroscience—an Asia-Pacific Strategic Enterprise), which targets the use of phase-contrast radiography to map one full human brain in a few years. Radiography was invented by Wilhelm Roentgen just a few hours after his discovery of X-rays ,2. To prSuch opportunities, however, remained largely unexploited for about one century. The detection of wave-like phenomena when electromagnetic radiation interacts with objects requires a special property called “coherence” ,4,8. We Starting in the 1960s, unconventional synchrotron radiation X-ray sources became available, which had high coherence levels ,4. CoherThese facts notwithstanding, even today imaging with coherent X-rays is practiced only by a small portion of its potential user community. This limitation is partly caused by an insufficient understanding of the techniques, their foundations, and their effectiveness. Unfortunately, most descriptions of coherence-based radiography cannot be understood without a rather sophisticated physics background ,9.The primary objective of the present text is to fill this gap. We begin with a very simple description of coherence-based radiography, avoiding formalism and using elementary examples to introduce the essential notions and techniques. Then, we illustrate the practical use of such techniques with a few concrete cases. A discussion will follow, again in simple terms, of some advanced coherence-based imaging strategies. Finally, we illustrate the general potential impact of coherent radiography by presenting the recently inaugurated international project SYNAPSE , which tA complete treatment of the image formation process in radiography would require a detailed theory of X-ray scattering by the electrons in the objects ,4. One wOne could thus imagine the retrieval of the charge distribution from the images by inverse Fourier transforms, and from it, the derivation of the structural characteristics of the objects. However, the practice of radiography is very different from this idealized view, for both fundamental and practical reasons. In essence, the vast majority of radiography applications still rely only on absorption and on the visual inspection of the corresponding images. As mentioned, coherence-based imaging is used only, in a rather limited way, by a small community of specialists. When trying to export it beyond this restricted group, one cannot expect the audience to accept complex presentations. The objective of most clinical radiologists is just to “see” things in the objects, with as little processing and interpretation as possible. In their eyes, coherence-based techniques are justified only if they facilitate the observation of interesting features.And this is often true, as shown by the examples of Therefore, our first objective here is to understand how coherent X-rays produce the edge enhancement. The impact of this enhancement is vividly illustrated by The phenomena that improve the second image of Eo, propagating with the time t in the x-direction:λ is the wavelength and v is the velocity. To introduce the idea of “phase”, we shall consider the following, the simple mathematical form of an electromagnetic plane-wave. This is a sinusoidal electric field of amplitude π/λ)(x − vt) of the sine function in Equation (1). To verify that this function does propagate with time, note that E is the same for combinations of x and t that give the same phase. Consider for example the phase for x = zero, t = zero, which is zero. And the phase for x = Δx, t = Δt, which remains equal to zero if Δx − vΔt = zero, i.e., if v = Δx/Δt. This shows that the wave of Equation (1) propagates indeed with speed v. The “phase” of the wave is the argument  are in the 10−4–10−5 range or even smaller [What makes the refraction edge fringe pairs detectable in practice, as seen in  smaller .α depends on the edge geometry and is typically not far from 1. A simple but tedious trigonometric calculation  shows thδ correspond to deviation angles in the 10−4–10−5 radians range: for a detector-object distance ro ≈ 1–10 m, they would give refraction fringe widths of the order of 10–1000 µm. So, the above values of This is in agreement with the empirical findings and with the typical detector spatial resolution, explaining why refraction edge enhancement can be observed. Note that 10–1000 µm is also the typical width of the edges for which refraction enhancement can be seen.λ. More realistically, however, the beam consists of a band of wavelengths of width δλ. In principle, this could decrease or even wipe out the edge enhancement caused by refraction.To start understanding X-ray coherence and its role in edge enhancement, let us consider the wavelengths of the X-ray beam in λ defines the X-ray beam coherence. More specifically, it determines what is called the “longitudinal coherence”.This is expressed by saying that the bandwidth δλ and the edge enhancement, consider that the wavelengths do influence refraction, since δ in Equations (2) and (3) changes with them (see δ makes (Equation (3)) the deviations μ negligible and may impede the refraction edge enhancement. To understand the relation between the “longitudinal coherence” parameter δthem see . For exaλ ≈ 1 Å and with a bandwidth δλ = 1 Å, extending from 0.5 to 1.5 Å. We see in δ roughly range from 10−6 to 10−5. Suppose that refraction enhancement applies to an edge of width w = 10 µm with a slanting angle α = π/4. The optimized distance ro for the central wavelength λ ≈ 1 Å—roughly corresponding to δ ≈ 2 × 10−6 and (Equation (3)) to μ ≈ 2 × 10−6 radians—is ro ≈ w/μ ≈ 5 m. Consider a typical synchrotron X-ray beam for imaging, centered at ro, the other wavelengths in the band 0.5–1.5 Å and their δ-values. Short wavelengths, 0.5–1 Å, correspond to narrower fringes than for the central wavelength, but to the same fringe positions. If anything, this increases the fringe visibility. Consider now, for this optimized Long wavelengths in the band, 1–1.5 Å, correspond instead to broader fringes with less visibility. However, such wavelengths are more strongly absorbed by the material and contribute less to the fringes. These intuitive arguments, corroborated by abundant empirical evidence, reveal a fundamental fact : the finλ by filtering synchrotron radiation with additional devices, the so-called “monochromators”. This is very good, since monochromators sharply decrease the X-ray flux. Avoiding them strongly increases the signal in the imaging experiment. Historically, this played a crucial role in the expansion of phase-contrast radiography [Therefore, one does not need to decrease δiography . These facts can be summarized by saying that synchrotron radiation possesses sufficient “longitudinal coherence” for refraction edge enhancement. However, this is only one of the two aspects of coherence. The second one is “lateral coherence”, which is related to the geometry of the X-ray source. To introduce the notion of “lateral coherence”, note that an excessively large source size in We shall further elaborate on “lateral coherence” later. But we can immediately realize that for phase-contrast imaging the X-rays must possess sufficient levels of both kinds of coherence, longitudinal and lateral. And this is true for the synchrotron sources of today without requiring further measures like monochromatization. Let us now go back to ro is large enough (right-hand side of the figure), one also sees a fringe sequence like that in the bottom part of If absorption was the only active mechanism, then the detected image would simply consist of two regions with different intensities, as seen on the left-hand side of Elementary optics shows  that thero << ρo, where ρo is the distance between the X-ray source and the object. By deriving it, we can discover the origin of Fresnel edge diffraction.Specifically, Equation (4) is valid for the realistic case λ. And dark fringes occur when the path difference equals an odd-integer multiple of λ/2. Consider, in fact, a ray that reaches the detector after having been scattered by the edge. On the detector, its wave is combined with that of the rays that reach the same point without being scattered. Constructive interference, corresponding to bright fringes, occurs when the path difference equals an integer number of wavelengths ro. For a ray scattered by the edge and arriving at the position z on the detector, the path is (ro2 + z2)1/2.The object-detector path for rays without scattering is z = Δz, is λ/2. Thus:λ << ro, gives: The path difference for the first bright fringe is zero. And that for the adjacent dark fringe, occurring at The practical detection of a Fresnel diffraction fringe sequence is possible in practice only under certain conditions. These include sufficient spatial resolution and high coherence. z. From Equation (4), we see that Δz increases with ro. Thus, one can again compensate for the limited spatial resolution of the detector by placing it farther away from the object.Specifically, the spatial resolution of the detector must be sufficient to distinguish adjacent fringes, i.e., it must be better than Δλ in Equation (4). Assume once again that the X-ray beam is not “monochromatic” but has a bandwidth δλ. This “blurs” the inter-fringe distance from Δz to δΔz and could make the fringes impossible to detect. Concerning “longitudinal coherence”, we must consider the presence of z does not exceed the inter-fringe distance itself: δΔz < Δz. From Equation (4) we get:The fringe sequence is still visible if the “blurring” δΔz < Δz is satisfied if:Thus, the requirement δΔEquation (7) is the condition for “longitudinal coherence” in this particular case.z-direction? In addition, there is also a source geometry condition for “lateral coherence”. So far, our analysis implicitly assumed an X-ray source with a small size. What happens if the source has a large transverse dimension, for example in the In essence, the emission of each source point produces an individual diffraction pattern. The superposition of all patterns can wash out the fringes, making it impossible to detect them. Hence, we find here again that “lateral coherence” requires a small source.ro is large enough to compensate for the limited spatial resolution of the detector. There is, however, an important difference between the two kinds of fringes. For a slanted edge, the fringe pair caused by refraction and the fringe sequence caused by diffraction can be combined. Both become visible if ro is increased. On the contrary, we see in ro and diffraction fringes for large values [The visibility of a diffraction fringe sequence continues to improve as e values . In the practice of coherence-based radiography, refraction fringe pairs are most frequently observed whereas diffraction fringe sequences are seldom seen. This is due to the fact that complicated edge shapes yield complex diffraction sequences, difficult to observe. Whereas refraction fringe pairs are typically simple and intense, and therefore easy to see.ro. The best approach is to boost the visibility of refraction fringe pairs, which are most effective in producing edge enhancement. Note again, however, that this optimization is not universal and must be repeated for each object.These empirical facts can be exploited to optimize the image quality by tuning The practical way to implement edge-enhanced radiography is by using the geometry of The simple online geometry of the top part of n is very close to 1 [The technical problem with this approach is finding suitable lenses for X-rays. The devices for visible light, such as focusing mirrors or refractive lenses, cannot be easily transferred to the X-ray wavelength range. We have seen that X-ray refraction is very weak, since the index of refraction ose to 1 . And theAs a consequence, special focusing devices must be invented and realized for X-rays: this is a crucial problem in the evolution of advanced radiography towards high spatial resolution. Major progress was made in recent years ,24,25,26The two top pictures a specifii-th zone equals i full wavelengths:Constructive interference at the focal point F occurs if the difference in the path between the rays passing through the center and the λ in Equation (8) implies that the typical size of X-ray FZPs is much smaller than for visible-light FZPs. Fabricating such minuscule devices is not easy [iMAX of i, as high as possible. The presence of not easy ,22,23,24MAX of the outermost zone:The same equation gives for the width ΔrMAX ≈ (λiMAXf)1/2, which gives iMAX ≈ rMAX2/(λf), so Equation (9) implies:Equation (8) shows that the FZP radius is rMAX requires a very narrow outermost zone width ΔMAX, which is very difficult to fabricate. This difficulty is augmented by the weak X-ray absorption of the blocking zones, which requires them to have a large thickness. Therefore, the outermost zones must be radially narrow but very thick, i.e., with a high “aspect ratio” [This is a key result: a large FZP radius t ratio” ,22,23,24The mechanical fragility of such features makes fabricating FZPs for X-rays a very challenging task. This was successfully implemented, as shown in One might be surprised that this lens works for X-rays, since it uses two reflections at near-normal incidence, which should be very weak at short wavelengths. The secret is the use of multilayer coatings that increase X-ray reflectivity. The feasibility of this approach was demonstrated by successful tests, as reported in Ref. .Finally, The combination of focusing/magnification with coherence-based imaging can be also improved by adding a phase-controlling element to an FZP microscope, as shown in the bottom part of Our previous discussion was focused on edge enhancement by refraction, which as mentioned dominates most applications of phase-contrast radiography. But other interesting coherence-based techniques exist that deserve at least a first look, as we shall discover in Let us start with a general empirical definition of coherence: “the property that makes it possible to detect wave-like phenomena”. To analyze its essential aspects, we can use anyone of such phenomena. Here, we shall select the simple case of diffraction by a narrow circular pinhole . This isλ, as shown in Pinhole diffraction produces a detectable pattern of circular bright-dark fringes only if the radiation has a sufficient level of coherence. What are the corresponding conditions? Let us start with a point-like source that emits only one wavelength λ, as in the right-hand image of roλ/d, where d is the pinhole diameter.Let us now relax these conditions, assuming that the source is not “monochromatic” but emits a wavelength bandwidth δλ “spreads” this inter-fringe distance to roδλ/d, blurring the fringe pattern. Which ceases to be visible when the spread becomes comparable to the distance between two adjacent fringes.The bandwidth δroδλ/d < roλ/d, or:Thus, the requirement for the pattern to be observable is This is in this case the condition for “longitudinal coherence”, qualitatively equivalent to that of Equation (7). Note that this condition is not very stringent. As a curiosity, applied to sunlight it explains why we can observe wave-like phenomena in everyday life, such as the beautiful colored fringes of soap bubbles. Indeed, the detection bandpass of our eyes guarantees that Equation (11) is satisfied for the solar spectrum that we can see. ξ and with an angular beam divergence corresponding to a solid angle Ω.In addition to “longitudinal coherence”, there is a “lateral coherence” (source geometry) condition for seeing the pinhole diffraction fringes. We assumed so far a point-like source, but ρo the distance between the source and the object, the solid angle corresponding to the pinhole area πd2/4 is ≈ (πd2/4)/ρo2. Therefore, the portion η of the emitted power that contributes to diffraction is:The angular divergence implies that only a portion of the source emission passes through the pinhole and participates in diffraction. Look at Ω decreases, η increases: a larger portion of the source emission contributes to the diffraction pattern, improving its signal-to-noise ratio. Conversely, if η decreases more of the emission is wasted as far as pinhole diffraction is concerned. As the divergence ξ of the source. Each source point acts as an emitter and produces a diffraction pattern centered at a specific position on the detector. The overall pattern is the superposition of all these individual patterns. And it could be no pattern at all, i.e., fringes may no longer be visible. Let us now bring into the picture  the finiξ(ro/ρo). Fringes are still visible if this distance is smaller than that between adjacent fringes, ≈roλ/d. So, the approximate condition to see the fringes is ξ (ro/ρo) < ro(λ/d), or:To analyze this point, note in ρoλ/ξ on the pinhole plane can contribute to diffraction, i.e., it is “laterally coherent”. Equation (13) can be interpreted as follows: only radiation emitted within an area of diameter η: By combining Equations (12) and (13), we discover an important upper limit for the parameter ηmax is called the “coherent power fraction” and measures how much “laterally coherent” radiation emits the source. In other words, it gauges the level of “lateral coherence”. The quantity ξ and/or the angular divergence Ω decrease. This illustrates the role of the source geometry in phase-contrast imaging. Note that the coherent power fraction, and therefore the “lateral coherence”, increases if the source size ξ of a synchrotron radiation X-ray source corresponds to the cross-section of the electron beam circulating in the accelerator. This cross-section, and therefore ξ, are typically very small thanks to the sophisticated accelerator technology of today [Practically speaking, the size of today ,4,9. TheΩ of synchrotron sources is also very small. This is a phenomenon caused by Einstein’s relativity, which produces laser-like beams of X-rays [Likewise, the angular divergence f X-rays ,4,9.λ. And are much more difficult to satisfy for short X-ray wavelengths than for visible light. This explains why coherence-related applications started for X-rays only in the recent decades, whereas they have been present for centuries in visible optics.In essence, synchrotron X-ray sources reach high levels of longitudinal and lateral coherence, fully suitable for phase-contrast radiography. This solved a big historical problem. Both the conditions for “longitudinal coherence” (Equations (7) and (11)) and for “lateral coherence” (Equation (14)) depend on the wavelength ξ. Before concluding our elementary discussion, we must discover a fundamental limit affecting “lateral coherence”. Consider a hypothetic procedure  to artifξ decreases. Elementary optics shows that the minimum value of the product ξ2Ω equals λ2. Thus, the coherent power fraction ηmax defined by Equation (14) cannot exceed the value π/4.However, pinhole diffraction increases the angular divergence of the radiation as π/4 cannot be surpassed by improving the instrumentation. When a source reaches this limit, it has full “lateral coherence”. This is a fundamental property of physics, called “diffraction limit”. The maximum value Many of the synchrotron sources of today reach the diffraction limit at least in part of their emitted spectrum of wavelengths. This is true, in particular, for the new class of X-ray sources called “free-electron lasers” . Concrete applications of coherence-based radiography impact many different areas of science and technology. The examples are becoming very numerous: selecting a subset is not trivial. We shall concentrate here on some of our results in materials science, the life sciences, and cultural heritage, selecting examples that provide a good idea of the power and flexibility of the novel radiography techniques.δ (defining the refractive index n with Equation (2)) and β, the absorption coefficient. Before introducing such examples, we can anticipate that the dominating mechanism for all of them is once again refraction-induced edge enhancement. Absorption also contributes to the images, but its effects are weaker. This can be understood from δ and absorption for β—are limited. We see, however, that the effects of absorption are comparatively weaker. So, by not exploiting contrast mechanisms other than absorption one would miss excellent research opportunities, as clearly illustrated by the following examples.Both parameters are small, so the corresponding effects—refraction, interference, and diffraction for Edge-enhanced radiographs taken with non-monochromatized synchrotron radiation reach excellent signal-to-noise ratios and contrast levels. This can be exploited to detect minute object details that are practically invisible in conventional absorption radiography. Furthermore, it opens the door to dynamic experiments, which image-evolving systems on short time scales.Tsai, W. L. et. al. (1996) reports a relevant example of such time-evolution studies . The invIt is known, for example, that in some cases electrodeposition of metal coatings yields poor results: visually unattractive rough-looking surfaces. A mechanism was proposed long ago to explain these failures and seek a remedy. But the hypothesized phenomenon was difficult for many to be accepted as plausible—until coherent X-ray radiography demonstrated its existence with real-time movies.During electrodeposition, gas bubbles are formed close to the surface that is coated. The hypothesis was that the metal overlayer, under certain conditions, does not initially form on the substrate itself but on the surfaces of the bubbles. These then dynamically disappear, leaving cavities at the substrate-coating interface. Such leftover cavities can cause the poor quality of the coating.Experimental tests with phase-contrast movies demonstrated the validity of this hypothesis in a very direct way. Edge enhancement by refraction is indeed ideal to visualize bubbles. And metal coatings strongly absorb X-rays, becoming very visible. The combination of these features allowed taking real-time movies of the process, as shown in The results were unmistakable and illustrated the power of this type of X-ray imaging. The hypothesized phenomenon was clearly visible and could be analyzed in detail, in view of a possible optimization of the deposition parameters to avoid defects. No matter how hard to believe, the “building on bubbles” mechanism could no longer be doubted after being directly seen . No dataVessels in living organisms are particularly difficult to image with conventional absorption-contrast radiography. In fact, the contrast is particularly weak between different organic tissues. This has very important consequences. For example, radiographic screening of heart diseases related to coronary arteries is still quite difficult, invasive, and not widely used.The negative consequences of limited contrast also impact life science research. Indeed, it makes it difficult to detect small vessels, which often play a fundamental role in physiological mechanisms.Tsai, Y. L. et al. (2002) presents a particularly challenging example of this problem . The expThe emitted power is indeed astonishing. If scaled up to a flying plane, the emission would likely cause it to crash. Thus, nature implements in fireflies an exceedingly effective light-producing process that could inspire human-oriented technologies.S)-2--4,5-dihydrothiazole-4-carboxylic acid, also known as “firefly luciferin”. This requires supplying oxygen, but until recently it was not yet fully clear how this is accomplished in the firefly organ “lantern”. Part of this organ is visualized in The phenomenon is not yet entirely explained and different hypotheses have been proposed for it. The emission takes place by breaking the compound . As it is well known, CAT produces three-dimensional radiographic pictures of objects with excellent flexibility in selecting the viewing mode. Thus, it has become a very widely used instrument for medical diagnostics.The foundation of CAT is the three-dimensional information present in a set of two-dimensional (“projection”) radiographic images taken from different viewpoints. A typical CAT set consists of several thousand “projection” pictures acquired in different directions, by rotating the radiography apparatus or, alternatively, the object. Pictures are normally taken at regular angular intervals.From a CAT set, well-established computer algorithms can reconstruct virtual images of different object parts, seen from different points of view and with adjustable depth penetrations. Over the past several decades, absorption-based CAT has become a very sophisticated and very powerful instrument.The advent of phase contrast in X-ray imaging triggered the obvious question: could it be implemented in the CAT mode? The issue was not trivial, since the information yielded by a “phase object” is not equivalent to that from an “absorption object”. And the CAT reconstruction techniques had been specifically developed for absorption images. Could they be applied to phase-contrast images, and if yes to what extent?Once again, practical tests provided the answer. Phase-contrast tomography was successfully implemented in many cases, delivering useful and rather spectacular results ,28,38,39Phase-contrast tomography has become by now a routine technique. Its main practical problem is the need to take large sets of projection images, which increases the duration of the experiment. As we shall see in the last section of this text, excellent progress has been recently made to tackle this issue.Our previous discussion, mostly limited to X-ray refraction and/or absorption, was somewhat oversimplified. In fact, several other phenomena contribute to X-ray image formation. The corresponding, valuable information can be extracted with experimental strategies different from those described so far.For example, monochromatized synchrotron radiation can analyze phenomena occurring at specific wavelengths. And sophisticated techniques for spatial and/or spectral filtering of the X-rays can separate the different contributions to the images.One very relevant example is “Diffraction Enhanced Imaging” (DEI) ,40,41. WWhat will be the long-term impact of coherence-based radiography? Difficult to predict, for a number of reasons. But to have any impact at all, the new techniques must be massively applied to real problems in science and technology. Unfortunately, there exists in this domain a certain tendency to favor the development of experimental techniques over their practical applications. This is not good: unused techniques, no matter how clever and sophisticated, are of no value. And it is not easy to transform new techniques from mere feasibility tests to practical tools.This transformation is further complicated if the objective is medical diagnostics. In fact, such an objective is affected by many other technical and non-technical issues. The efforts towards broader practical applications are easier—and desirable—for materials science and biology. We shall illustrate these points with a specific and very relevant project: SYNAPSE, whose objective is to map large animal and human brains. Its ambition is to obtain comprehensive charts of all neuron cells and of their connections. One of the key achievements prior to SYNAPSE ,43 was mThe development of practical strategies to transform such a dream into reality reveals the formidable obstacles that separate feasibility tests from real-life experiments. These are the most important.Spatial resolution: the radiography techniques must be able to detect details of the objects on a scale suitable for brain mapping. One must consider two resolution levels. At the micron scale, the edge-enhancement techniques previously described are adequate and yield good general images of brain cells with some details. However, regions of special interest must be imaged with a better, nanometer-scale resolution. This is achieved with the microscopy techniques discussed in Total duration of the experimental program: mapping the Drosophila brain required a few years [ew years ,43. Mostew years  to engagImage contrast: microscopic soft tissue features in brain specimens such as organelles, axons, and synapses do not offer much contrast for X-ray imaging, even in the presence of edge enhancement. So, the contrast must be boosted by staining the objects [ objects ,43. The  objects . Radiation damage: this is a potential issue for all radiological studies at the cellular level. The problem is typically handled with empirical approaches, systematically checking the specimens for possible modifications. So far, the results have been negative, but systematic assessments are required.Data processing: the project of human brain mapping is primarily based on tomography and requires computer reconstructions of very many large tomographic image sets. The amount of data to be processed for SYNAPSE is enormous: it would largely exceed the present resources of organizations like Google. Once again, a coordinated effort over different countries appears necessary [ecessary . Data storage, availability, and security: the objective of SYNAPSE is to make its data available to all qualified researchers worldwide. But the sheer dimension of the information sets creates extreme problems. Access should be guaranteed without barriers and across national borders. But this creates formidable problems of data security. Once again, there is a big gap between feasibility tests and real experiments that handle real data.How can all these challenges be managed? SYNAPSE  is a conAt this time, SYNAPSE involves Japan, South Korea, China, Australia, Taiwan, and Singapore, with foreseen extensions to other countries. Its present partners are PAL  RIKEN/Spring-8, Shanghai/IoN , SSLS/NUS , Yong Loo Lin School of Medicine (NUS), the Institute of Physics of the Academia Sinica , NSSRC  and the Australian Synchrotron .SYNAPSE is built on the success of the previous program AXON (Accelerated X-ray Observation of Neurons). Which produced, in particular, important technical results as far as accelerating the image taking is concerned ,42,43. The SYNAPSE coalition was formally inaugurated early in 2020 with a ceremony in Singapore. In spite of the unforeseen problems caused by the Covid-19 pandemics, the programs initiated and already produced good results . The objSYNAPSE is also tackling the huge problems of data processing, storage, security, and availability. This is handled by a consortium of high-performance computing institutions in the participating countries, under the leadership of the National Supercomputer Center in Singapore .


PAC) reinforcement learning (RL) algorithm for Markov decision processes (MDPs) that intelligently maintains favorable features of both model-based and model-free methodologies. The designed algorithm, referred to as the Dyna-Delayed Q-learning (DDQ) algorithm, combines model-free Delayed Q-learning and model-based R-max algorithms while outperforming both in most cases. The paper includes a PAC analysis of the DDQ algorithm and a derivation of its sample complexity. Numerical results are provided to support the claim regarding the new algorithm’s sample efficiency compared to its parents as well as the best known PAC model-free and model-based algorithms in application. A real-world experimental implementation of DDQ in the context of pediatric motor rehabilitation facilitated by infant-robot interaction highlights the potential benefits of the reported method.This paper offers a new hybrid probably approximately correct ( RL) algorithms can apply to a dynamical system modeled as a Markov decision process (MDP), few are probably approximately correct (PAC)—meaning able to guarantee how soon the algorithm will converge to a near-optimal policy. Existing PACMDP algorithms can be broadly divided into two groups: model-based algorithms like  with a model-based (R-max) PAC algorithm, in order to give rise to a new PAC algorithm (Dyna-Delayed Q-learning (DDQ)) that combines the advantages of both.Typically, model-free algorithms circumvent the model learning stage of the solution process, a move that by itself reduces complexity in problems of large size. In many applications, however, model learning is not the main complexity bottleneck. Neurophysiologically-inspired hypotheses  have sugRL algorithms begins with the Dyna-Q algorithm  provides a smooth (er) transition from model-free to model-based, during the learning process . This is also distinct from another thread of research that addresses uncertainty and robustness in MDPs whose parameters are randomly (or even adversely) selected from a set and can vary over the instances when the same state-action pair is encountered  did not result in models of particularly large state and action spaces, but were particularly complicated by the absence of sufficient data sets for learning (HRI data can be collected) is usually limited to a few hours per month. There is a need, therefore, for reinforcement learning approaches that can maintain (or even guarantee) efficiency and accuracy even when the learning set is particularly small.Our own motivation for developing of this new breed of learning . This isPACRL algorithm in the form of a well-known theorem. The DDQ algorithm is introduced in PAC properties, and the analytic derivation of its sample complexity. Finally, DDQ outperforms its parent algorithms as well as the state-of-the-art model-base and model-free algorithms in terms of the required samples to learn near-optimal policy. Experimental results from application of DDQ in the context of early pediatric motor rehabilitation suggest the algorithm’s efficacy and its potential as part of a child-robot interface mechanism that involves autonomous and adaptive robot decision-making. To enhance this paper’s readability, the proofs of most of the technical lemmas supporting the proof of our main result are moved to the paper’s Appendix.The paper starts with some technical preliminaries in MDPM is a tuple {S, A, R, T, γ} with elementsS a finite set of statesA a finite set of actionsR: S × A →  the reward from executing a at sT: S × A × S →  the transition probabilitiesdiscount factorA finite policyπ is a mapping π: S → A that selects an action a to be executed at state s. A policy is optimal if it maximizes the expected sum of discounted rewards; if t indexes the current time step and at, st denote current action and state, respectively, then this expected sum is written γ here reflects the preference of immediate rewards over future ones. The value of state s under policy π in MDPM is defined asA state-action pair  under policy π:Note that an upper bound for the value at any state is MDPM has at least one optimal policy π* that results in an optimal value  assignment at all states; the latter is denoted Every The standard approach to finding the optimal values is through the search for a fix point of the Bellman equationRL) is a procedure to obtain an optimal policy in an MDP, when the actual transition probabilities and/or reward function are not known. The procedure involves exploration of the MDP model. An RL algorithm usually maintains a table of state-action pair value estimates Q  that are updated based on the exploration data. We denote Qt  the currently stored value for state-action pair  at timestep t during the execution of an RL algorithm. Consequently, vt(s) = maxaQt . An RL algorithm is greedy if it at any timestep t, it always executes action at = arg maxa∈AQt . The policy in force at time step t is similarly denoted πt. In what follows, we denote |S| the cardinality of a set S.Reinforcement learning , and the resulting estimated model subsequently informs the computation of the optimal policy. A model-free RL algorithm, on the other hand, would skip the construction of an estimated MDP model, and search directly for an optimal policy over the policy space. An RL algorithm is expected to converge to the optimal policy, practically reporting a near-optimal one at termination.Reinforcement learning algorithms have been classified as PAC) analysis of RL algorithms deals with the question of how fast an RL algorithm converges to a near-optimal policy. An RL algorithm is PAC if there exists a probabilistic bound on the number of exploration steps that the algorithm can take before converging to a near-optimal policy.Probably approximately correct , and letKt ⊆ S × AMDPbe a set of state-action pairs labeled known. The known state-action is anMDPderived fromMandKtby defining new stateszs,afor each unknown state-action pair ∉Kt, with self-loops for all actions, i.e.,. For all  ∈ Kt, it isand. When an unknown state-action pair ∉Ktis experienced,and the model jumps tozs,awith; subsequently,.Kt be set of current known state-action pairs of an RL algorithm t, and allow Kt to be arbitrarily defined as long as it depends only on the history of exploration data up to t. Any ∉Kt experienced at time step t marks an escape event.Let Theorem 1Letbe a greedyRLalgorithm for an arbitraryMDPM, and letKtbe the set of current known state-action pairs, defined based only on the history of the exploration data up to timestept. Assume thatKt = Kt+1unless an update to some state-action value occurs or an escape event occurs at timestept, and thatQt ≤ vmaxfor all  andt. Letbe the known state-actionMDPat timesteptandπt(s) = arg maxaQt denote the greedy policy thatexecutes. Suppose now that for any positive constantϵandδ, the following conditions hold with probability at least 1 − δfor alls,aandt:. LetAoptimism:accuracy:complexity:sum of number of timesteps withQ-value updates plus number of timesteps with escape events is bounded byζ > 0. Then, executing algorithmon anyMDPMwill result in following a 4ϵ-optimal policy on all buttimesteps, with probability at least 1 − 2δ.DDQ and the main contribution of this paper. Ddq integrates elements of R-max and Delayed Q-learning, while preserving the implementation advantages of both.This section presents Q-learning algorithm  Linelgorithm  with a mtype-1 update (model-free update), and to the one on line 52 as a type-2 update (model-based update). The latter offers a way for new model-related information to be injected into the model-free learning process. Type-1 updates use the m1 most recent experiences (occurances) of a state-action pair  to update that pair’s value, while a type-2 update is realized through a value iteration algorithm (lines 43 − 54) and applies to state-action pairs experienced at least m2 times. The outcome at timestep t of the value iteration for a type-2 update is denoted ϵ2 regulates the desired accuracy on the resulting estimate : the number of samples gathered for the update type-1 of Q  once l  = m1.• U : the running sum of target values used for a type-1 update of Q , once enough samples have been gathered.• b : the timestep at which the most recent or ongoing collection of m1  experiences has started.• Q . The flag is set to true initially, and is reset to true whenever some Q-value is updated. It flips to false when no updates to any Q-values occurs within a time window of m1 experiences of  in which attempted updates type-1 of Qi  fail.• n : variable that keeps track of the number of times  is experienced.• n : variable that keeps track of the number of transitions to s′ on action a at state s.• r : the accumulated rewards by doing a in s.• We refer to the assignment in line 31 of Algorithm 1 as a estimate . A type-DDQ algorithm is tuned via the m1 and m2 parameters. One can practically reduce it to Delayed Q-learning by setting m2 very large, and to R-max by setting m1 large. The next section provides a formal proof that DDQ is not only PAC but also possesses the minimum sample complexity between R-max and Delayed Q-learning in the worst case—often, it outperforms both.The execution of the Q-learning is incomparable .Before formally stating the Definition 3An event whenand at the same timel = m1orn = m2, is called an attempted update.Definition 4At any timesteptin the execution ofDDQalgorithm the set of known state-action pairs is defined as:s, a) known, the set Kt will be partitioned into two subsets:In subsequent analysis, and to distinguish between the conditions that make a state-action pair , timesteptis called a successful timestep for  if atteither update type-1 happens toQ or the number of times that  is visited reachesm2.ϵ1. Defining rewards as positive quantities prevents the Q-values from becoming negative. At the same time, state-action pairs can initiate update type-2 only once they are experienced m2 times. Together, these conditions facilitate the establishment of an upper-bound on the total number of successful timesteps during the execution of DDQ:Recall that a type-1 update necessarily decreases the Q-value by at least Lemma 1The number of successful timesteps for a particular state-action pair  in aDDQalgorithm is at most. Moreover, the total number of successful timesteps is bounded byκ.Proof. See Lemma 2The total number of attempted updates inDDQalgorithm is bounded by |S‖A|(1 + κ).Proof. See Lemma 3LetMbe anMDPwith a set of known state-action pairsKt. If we assume that for all state-action pairs ∉Ktwe have, then for all state-action pairs in the known state-actionMDPit holdsProof. See m1 big enough and applying Hoefding’s inequality allows the following conclusion  experiences a successful update of type-1 with its value changing fromQ toQ′, and that there existssuch that∀s ∈ Sand∀t′ < t,. Ifforκ = |S‖A|(1 + 1/(1 − γ)ϵ1), thenwith probability at least.Proof. In m2, and the number of iterations for the value iteration part of the DDQ algorithm in order to obtain a desired accuracy.The following two lemmas are borrowed from  with verLemma 5Proposition 4)) Suppose the value-iteration algorithm runs onMDPMforiterations, and each state-action value estimateQ is initialized to some value between 0 andvmaxfor all states and actions. LetQ′ be the state-action value estimate the algorithm yields. Then. estimate ofRandT, withn ≥ m2, in the form ofandrespectively. WithCa constant and for all state-action pairs, choosingwith probability at least. Moreover, for any policyπand for all state-action pairs,with probability at least.Proof. Combine  ∈ S × A, we have:with probability at least.Proof. See DDQ; however, for the complexity condition of Theorem 1 to be satisfied, one needs to show that during the execution of m1 as in Lemma 9With the choice ofm1as in, and assuming theDDQalgorithm at timesteptwith ∉Kt,l = 0 and, we know that an attempted type-1 update ofQ will necessarily occur withinm1occurrences of  aftert, say at timestep. If  has been visited fewer thanm2till, then the attempted type-1 update atwill be successful with probability at least.Proof. See Lemma 10Lett be the timestep when  has been visited form1times after the conditions ofwere satisfied. If the update at timesteptis unsuccessful and at timestept + 1 it is, then  ∈ Kt+1.Proof. See DDQ algorithm can be derived in a straightforward way. Note that a state-action pair that is visited m2 times becomes a permanent member of set Kt. Therefore, the number of escape events is bounded by |S‖A|m2. On the other hand, DDQ as the minimum among the two bounds.A bound on the number the escape events of ism i.e.  suggest Lemma 11During the execution ofDDQ, with the assumption thatholds, the total number of timesteps with ∉Kt (i.e. escape events) is at most.Proof. See PAC properties of the DDQ algorithm and provides a bound on its sample complexity.Next comes the main result of this paper. The statement that follows establishes the Theorem 2Consider anMDPM = {S, A, T, R, γ}, and let, andδ ∈ . There existandwithand, such that ifDDQalgorithm is executed,Mfollows a 4ϵ-optimal policy with probability at least 1 − 2δon all butoptimism condition, we start by proving that Proof. We intend to apply t = 1, the value of all state-action pairs are set to the maximum possible value in MDPM. This implies that t = n − 1. 3) At timestep t = n, all t = n. For these state-action pairs, 1) At For all M exept for t and all state-action pairs, which directly implies Note that accuracy condition, first writeTo establish the s, a) ∈ Kt, there can be two cases: either β ≤ 3ϵ1. If If  ∈ Kt, β ≤ 3ϵ1 with probability at least a = πt(s) and mirroring α = 0  is bounded by κ + min  with probability Finally, to analyze δ and therefore the DDQ algorithm is PAC. Substituting ζ into In conclusion, the conditions of DDQ algorithm to its parent technologies. It proceeds with additional comparisons to the state-of-the-art in both model-based 6 , the lowest bound on the sample complexity is |S‖A|/ϵ2 (1 − γ)7, achieved by UCB Q-learning (MDP).The lowest known bound on the sample complexity of a model-based lgorithm ). For thlearning   deterministically transports the agent to state + with reward 0.5 + ϵ′ (with ϵ′ > 0). Let a be any of the other A− 1 actions (represented by dashed lines). From any state i ∈ {1, …, N}, taking action a will trigger a transition to state + with reward 1 and probability pia, or to state − with reward 0 and probability 1 − pia, where pia ∈ {0.5, 0.5 + 2ϵ′} are numbers very close to 0.5 + ϵ′. For each state i ∈ {1, …, N}, there is at most one a such that pia = 0.5 + 2ϵ′. Transitions from states + and − are identical; they simply reset the agent to one of the states {1, …, N} uniformly at random.To perform a fair and meaningful comparison of these algorithms to MDP such as the one shown in i ∈ {1, …, N} is independent of the other states; specifically, it is the action marked by the solid arrow if pia = 0.5 for all dashed actions a, or the action marked by the dashed arrow for which pia = 0.5 + 2ϵ′, otherwise. Intuitively, this MDP is hard to learn for exactly the same reason that a biased coin is hard to be recongized as such if its bias  is close to 0.5  policy of M in terms of the required samples. As expected, the required samples decrease  in m (m = 600). For that reason, we cap m at 600 which requires 7,770 samples on average for Mormax to learn the optimal policy. Yet another important performance metric to record for a model-based RL algorithm is the number of times it needs resolve the learned model through value-iteration, since the associated computational effort is highly dependent on this number. For Mormax, the average number of times it performs model resolution is 12.06.We empirically fine-tune the parameters of Mormax and UCB Q-learning algorithms to maximize their performance on learning the near optimal (4ly) in m  until thc2 parameter. The value of c2  is the most common Markovian model because some internal parameters of the human partners such as intent are not directly observable    or an MDDP)  part of six 1-h infant-robot social interaction sessions with a 10 month old subject, and data in the form of video are collected and annotated. In the six sessions the robot was remotely controlled and its actions were chosen by a human operator who was observing the interaction. The I which is a random variable that indicates the improvement as a result of using DDQ optimal policy and is expressed as DDQ algorithm is used (as it was in last two rehabilitation sessions), and t-test with releasing the constraint of equal standard deviation for the two group , one can confidently attest that it is possible that a DDQ policy can outperform a human-driven social interaction strategy.To put the figures of wo group  in orderRL algorithm that captures favorable features of both model-based and model-free learning and most importantly preserves the PAC property can not only alleviate the cognitive load and time commitment of human caregivers when socially interacting in play-based activities with infants who have motor delays, but potentially also improve motor rehabilitation outcomes. One such algorithm which has been implemented and pilot-tested within an enriched robot-assisted infant motor rehabilitation environment is the DDQ. The DDQ algorithm leverages the idea of earlier Dyna-Q algorithms to combine two existing PAC algorithms, namely the model-based R-max and the model-free Delayed Q-learning, in a way that achieves the best (complexity results) of both. Theoretical analysis establishes that DDQ enjoys a sample complexity that is at worst as high as the smallest of its constituent technologies; yet, in practice, as the numerical example included suggests, DDQ can outperform them both. Numerical examples comparing DDQ to the state of the art in model-based and model free RL indicate advantages in practical implementations, and experimental implementation and testing of DDQ as it regulates a robot’s social interaction with an infant in a game of chase hints at possible advantages in rehabilitation outcomes compared to a reactive yet still goal-oriented human strategy.The design and implementation of an


Twisted Ring-LWE, which replaces the canonical embedding by an extended form. By doing so, we allow the Ring-LWE problem to be used over maximal real subfields of cyclotomic number fields. We prove that Twisted Ring-LWE is secure by providing a security reduction from Ring-LWE to Twisted Ring-LWE in both search and decision forms. It is also shown that the twist factor does not affect the asymptotic approximation factors in the worst-case to average-case reductions. Thus, Twisted Ring-LWE maintains the consolidated hardness guarantee of Ring-LWE and increases the existing scope of algebraic lattices that can be considered for cryptographic applications. Additionally, we expand on the results of Ducas and Durmus  on spherical Gaussian distributions to the proposed class of lattices under certain restrictions. As a result, sampling from a spherical Gaussian distribution can be done directly in the respective number field while maintaining its format and standard deviation when seen in Several works have characterized weak instances of the Ring-LWE problem by exploring vulnerabilities arising from the use of algebraic structures. Although these weak instances are not addressed by worst-case hardness theorems, enabling other ring instantiations enlarges the scope of possible applications and favors the diversification of security assumptions. In this work, we extend the Ring-LWE problem in lattice-based cryptography to include algebraic lattices, realized through twisted embeddings. We define the class of problems  Lattice-based cryptography comprehends the class of cryptosystems whose security is based on the conjectured intractability of hard lattice problems such as the Shortest Independent Vectors Problem (SIVP), the Shortest Vector Problem (SVP), and the Closest Vector Problem (CVP) ,2. The mn a power of two. The fact that R to its dual, denoted R, with no loss in their underlying worst-case hardness guarantees [Although the Ring-LWE hardness results hold for any number field ,9, its marantees .R considering a power basis, since the transformation to the associated vector subspace H isomorphic to H and computing the inverse transformation with respect to the canonical embedding [Another advantage of power-of-two cyclotomic number fields is that the sampling of error terms can be performed directly in the ring mbedding . For genmbedding ,12.There are several works in the literature exploring properties of number fields used in the foundation of some cryptosystems based on ideal lattices. An example is a quantum polynomial-time algorithm to find a small generator of a principal ideal in the ring of algebraic integers of cyclotomic rings , which ap prime, which was proposed for NTRU Prime [Although these weak instances are not addressed by worst-case hardness theorems , new proRU Prime , and sugRU Prime . In thistwisted embeddings. Since the canonical embedding is a special case of twisted embeddings, this replacement maintains the consolidated results for Ring-LWE. Twisted embeddings have been useful in coding theory, since they allow the construction of algebraic lattices with improved properties for Rayleigh fading channels, providing high density, maximum diversity, and great minimum product distance [In this context, we extend the Ring-LWE class of problems to embrace more general algebraic constructions of lattices which allow additional factors on the embedding coordinates. We replace the canonical embedding by distance ,34,35.Twisted Ring-LWE problem, in which the error terms are sampled in the space H isomorphic to We extend the Ring-LWE problem by replacing the canonical embedding with twisted embeddings on both the search and decision variants. As a result, we obtain the m is odd, and m is even, corresponds to a distribution with the same format in the space H, but linearly wider in the ring dimension. This occurs because the lattice obtained from the ring H. Examples of ideal lattices equivalent to p-th cyclotomic number fields. Since power-of-two cyclotomic rings have been widely used in cryptographic applications, we consider parameterizing the Ring-LWE problem with the ring of integers of the maximal real subfield of a cyclotomic number field. We discuss the limitations of using maximal real subfields in a public-key encryption scheme [H, their Ring-LWE instances can be efficiently converted between themselves.As a result, algebraic constructions from coding theory via twisted embeddings can also be used in cryptographic applications based on the Ring-LWE problem. In this work, we focused our attention on the algebraic construction of rotated d Durmus  showed tr fields , and ther fields , and then scheme  using thn scheme . Finallyn scheme . In factThis paper is organized as follows. n, the asymptotic factors are not affected by the change of embeddings.nd Regev , and alsm, denote by In this section, we introduce concepts, results and notation to be used throughout the paper. For a positive integer number space H.Frequently, lattices are defined in the Euclidean space  context ,9, it isDefinition 1.Let subspace is defined as(Space H)H endowed with the inner product obtained as a restriction of the standard inner product of We consider The norm .Let  canonical basis of H as an n-dimensional , if E with rank n. Equivalently, basis  as follows. Let n-dimensional distribution We extend this definition to elliptical Gaussian distributions in The smoothing parameter is a lattice parameter defining the width beyond which a discrete Gaussian starts to behave similarly to a continuous distribution . It is rDefinition 3.For an n-dimensional latticeΛand positive real  smoothing parameter (Smoothing parameter)n, normalizing the Gaussian function discrete Gaussian distribution over For any In this section, we summarize concepts and results from algebraic number theory, presenting as an example the case of cyclotomic number fields and their maximal real subfields. Details can be found in ,40.algebraic) number field K is a finite extension of the field K is a degree of K, denoted K. In general, if K and L are number fields such that An Example 2.For  maximal real subfield of K be a number field. A map involution of K if Let K, satisfies trivial (it is the identity); otherwise, the involution is said to be non-trivial. If K is its maximal real subfield [The subfield subfield .K be a number field of degree n. There are exactly n distinct monomorphisms (of fields) from K to Let real if complex. If n as K to canonical embedding from K into the subspace H is the homomorphismA monomorphism  problem ,9.signature of K. We say that K is totally real when K is totally complex when K is said to be a CM-field if it is totally complex and has degree two over its fixed field by the involution F [The pair lution F .Any cyclotomic number field K is said to be a Galois number field if, for every x over K. In this case, the set of automorphisms Galois group of K over K to The number field K be a Galois number field. For every trace and norm of any element Let K that are the root of a monic polynomial in ring of integers of K, denoted by K is a number field of degree n, its ring of integers has a n elements, which is called an integral basis of K. If The set of all elements in a number field n elements. The same holds if fractional ideal of K, which is a subset of K satisfying the condition that K, including its ring of integers, is a K. If m-th cyclotomic number field, then K is a monogenic number field.If codifferent ideal, that is, the dual ideal of the ring of integers. Frequently, the codifferent ideal is also denoted by different of K. In general, the dual ideal of any fractional ideal K is the fractional ideal K, defined asThe fractional ideal (x) of θ  -LWE hardness, namely the decision version of the Shortest Vector Problem (GapSVP), the Shortest Independent Vectors Problem (SIVP), and the Discrete Gaussian Sampling (DGS) problem, which is denoted  field K .Definition 4.Λ and length For an approximation factor LWE, the DGS problem can be stated as follows: given an K be a number field with ring of integers K, and let K, let In order to define the Ring-LWE distribution and the computational problems associated with it, let Definition 7.For  is the homomorphism (Twisted embeddings)F is totally positive, then Since sition 1 .Proposition 1.If M is a free .If M iK to K, then H.Twisted embeddings can be extended from n isometric to both Consider the natural extension of the trace function a under the canonical embedding are simply M of rank n can be seen as a full-rank lattice directly in the Euclidean vector space  (ii) Proof. By definition, Twisted Ring-LWE, or simply Ring-LWEIn this section, we propose an extended version of the Ring-LWE problem, adopting twisted embeddings rather than the canonical embedding. We refer to this new class of problems as  problem , providie is randomized by a distribution K a number field with an involution, F its associated fixed field, e is randomized by a distribution K.In the Ring-LWE distribution, the error Definition 11.For a totally positive element  Twisted Ring-LWE distributionwhere a is uniformly randomized in (Twisted Ring-LWE distribution)Analogously to Ring-LWE , which iy et al.  and the y et al. .Definition 12.For a positive real Definition 13.Let  Ring-LWE hardness consists in two reductions: Theorem 3.Let K be an arbitrary number field of degree n with ring of integers  [m is odd, and m is even. Then, the reduction modulo Durmus and Ducas proved a special case when a spherical Gaussian distribution with width eorem 4) . In ordem is odd and m is even. Also, let H be:In the statement of Theorem 4, let Theorem 4.Let  , and thes Q(ζ2k) , and therime p≥5 . The casrime p≥5 , and Ducrime p≥5 . In the Let Proposition 3Let , realized in Proposition 3, is given byThe generator matrix of the H.As an immediate consequence of Theorem 5, in Corollary 1 we prove that the construction for the Corollary 1.Let Proof. In the realization of the ven by M . Since Op prime, to analyze the practical impacts of instantiating the Ring-LWE problem over the ring of integers of K in the compact public-key cryptosystem of Lyubashevsky, Peikert, and Regev [In this section, we use the fact that nd Regev   for efficient polynomial multiplication. For example, for Notice that, in our case, c such that q in comparison with CRT-based algorithms for power-of-two cyclotomic number fields [expansion factor, a property introduced by Lyubashevsky and Micciancio [f isg after reduction modulo f. By computing the expansion factor of q such that the coefficients do not wrap around after arithmetic operations, avoiding the occurrence of decryption errors.Thus, d Seiler . Althougr fields ,46. Anotcciancio . The expf.In order to analyze the expansion factor of Lemma 4.If g is a polynomial in q in order to avoid the coefficients to wrap around after polynomial operations. This also leads to an increase in the length of system parameters and memory/bandwidth requirement for transmission of public parameters.For the case K does not increase as a power-of-two, one may want to find a ring instantiation that closely achieves a target security level. For example, to obtain a ring dimension between 700 and 800, the required for achieving 128-bit security [p ranges from the 223-th to the 252-th prime number, comprehending 29 possible choices.In the positive direction, since the dimension of security , possiblK is the maximal real subfield of a cyclotomic number field, whose dimension is K is the maximal real subfield of a cyclotomic number field. Moreover, we present future research possibilities related to the Twisted Ring-LWE problem.In a nutshell, we have discussed some practical impacts of instantiating the Twisted Ring-LWE problem when In this paper, we introduce an extension to the Ring-LWE class of problems, namely The Twisted Ring-LWE Problem ,9. The Rbeddings  rather tH via twisted embeddings. This generalizes the results of Ducas and Durmus in Theorem 5 [As a result, we broaden the scope of number of algebraic lattices that can be used for lattice-based cryptosystems, including those algebraic constructions of lattices that allow additional factors on the embedding coordinates. This type of construction has been useful in coding theory, since they allow the construction of algebraic lattices with improved properties for Rayleigh fading channels, providing high density, maximum diversity, and great minimum product distance ,34,35. Nheorem 5  and the p-th cyclotomic number fields. We analyze instantiating the Ring-LWE problem using maximal real subfields of p-th cyclotomic number fields in a public-key encryption scheme [p-th cyclotomic number fields grows exponentially.An example of a construction of the n scheme . By doinn scheme  and provH, without resorting to change of representation from Lyubashevsky, Peikert, and Regev  suggesteWe also leave as future work detailing how to connect Twisted Ring-LWE instantiations over different number fields, if the ring of integers of both number fields leads to equivalent lattices under twisted embeddings. By doing so, we can connect an instance on a power-of-two cyclotomic number field to an instance of a maximal real subfield as both rings of integers lead to a construction of the vidowitz . As a co


In this paper, we obtain the law of iterated logarithm for linear processes in sub-linear expectation space. It is established for strictly stationary independent random variable sequences with finite second-order moments in the sense of non-additive capacity. As fundamental limit theorems of probability theory, the classical law of the iterated logarithm (LIL for short) plays an important role in the development of probability theory and its applications. The original statement of LIL obtained by Khinchine  is for aAs is well known, the linear processes are especially important in time series analysis and they arise from a wide variety of contexts (cf. Hannan ). Applicc denotes a positive constant, which may take different values whenever it appears in different expressions.A natural question is: can LIL of linear processes be realized under Peng’s framework? The main purpose of this paper is to establish the law of iterated logarithm for linear processes generated by IID random variables in sub-linear expectation space. In the classical case, the LIL of partial sum is established by decomposing the linear process. We will find that this way is also valid for proving LIL for linear process in the sub-linear expectation space, though there are some differences. Intuitively, sub-linear expectation and related non-additive probabilities (Capacities) generated by them plays a decisive role in our proof. In the sequel, To state the result, we shall first recall the framework of sub-linear expectations. We use the framework and notation of Peng ,18,19. LDefinition 1.A function 1. Monotonicity: 2. Constant preserving: 3. Sub-additivity: 4. Positive homogeneity: The triple Remark 1.(i) The sub-linear expectation Definition 2.(i)  Let whenever the sub-expectations are finite. A sequence of random variables (ii) (Independence) In a sub-linear expectation space whenever (iii) (IID random variables) A sequence of random variables It is obvious that, if Next, we introduce the capacities corresponding to the sub-linear expectations.Definition 3.A set function V: 1. 2. .A set It is called to be sub-additive if A. ThenA sub-linear expectation V being replaced by In addition, a pair If of Zhang ).Definition 4.(a) A sub-linear expectation .(a) A (b) A function V: (c) A capacity V: c1. Continuity from below: c2. Continuity from above: It is obvious that the continuity from above and sub-additivity imply the continuity from below, and the continuity from the below and sub-additivity imply the countable sub- additivity. Therefore, we call a sub-additive capacity to be continuous if it is continuous from above. Set In this section, we shall study the LIL of linear processes under association assumption in the sub-linear expectation space. For any First, we give the definition of strictly stationary sequence under the sub-linear expectation.Definition 5.Next we give the main results: the law of the iterated logarithm for linear processes in the sub-linear expectation space.Theorem 1.Suppose that Define the linear process by where Remark 2.If Remark 3.In particular, according to Proposition 4.1 in Zhang , for theIn order to prove the main results, we need the following Lemmas. The first one was the convergence part of the Borel–Cantelli Lemma.Lemma 1.Let .Let {AThe second Lemma on the exponential inequality is Lemma 2.1 of Zhang .Lemma 2.Let where .Let {ZThe following Lemma is a law of iterated logarithm under sub-linear expectation.Lemma 3.Let Proof. Obviously, (4) can be directly derived from Theorem 3.11 and (4.29) in Zhang . □Lemma 4.Let Proof. Note that to prove , it suffBy the definition of Let now that∑k=1n−E^[ing that∑k=1nεk≤∑x, where First, to estimate I2, by , we get<1. Thengϵ(·)∈Cl,I2, by , a, a21), a<∞,thensupn and are also a natural generalization of the LIL under classical additive probability. The key to the main results of this paper is an exponential inequality. If we can establish the corresponding exponential inequalities for negative dependent (ND) sequences, then we can obtain the LIL of linear processes generated by stationary ND sequences under sub-linear expectation. ND sequences are weaker than independent sequences. Therefore, it is an impending problem to study the theoretical properties of ND sequences in sub-linear expectation, which is the subject of future research.