diff --git a/.DS_Store b/.DS_Store index 164fb3a86e66e8f6dc8b408f95571317d2a1922e..a7ef6fc0bc7282d5fcb6a848572066d73c5829ec 100644 Binary files a/.DS_Store and b/.DS_Store differ diff --git a/extensions/.DS_Store b/extensions/.DS_Store index 3f1d319698455eecac8b8f1fb59df47610c96850..27719949c135aae1faed48a9dc8abc55a4264719 100644 Binary files a/extensions/.DS_Store and b/extensions/.DS_Store differ diff --git a/extensions/adetailer/CHANGELOG.md b/extensions/adetailer/CHANGELOG.md new file mode 100644 index 0000000000000000000000000000000000000000..af8a888d28020932ac08da4ecdaa672891581c43 --- /dev/null +++ b/extensions/adetailer/CHANGELOG.md @@ -0,0 +1,235 @@ +# Changelog + +## 2023-07-07 + +- v23.7.4 +- batch count > 1일때 프롬프트의 인덱스 문제 수정 + +- v23.7.5 +- i2i의 `cached_uc`와 `cached_c`가 p의 `cached_uc`와 `cached_c`가 다른 인스턴스가 되도록 수정 + +## 2023-07-05 + +- v23.7.3 +- 버그 수정 + - `object()`가 json 직렬화 안되는 문제 + - `process`를 호출함에 따라 배치 카운트가 2이상일 때, all_prompts가 고정되는 문제 + - `ad-before`와 `ad-preview` 이미지 파일명이 실제 파일명과 다른 문제 + - pydantic 2.0 호환성 문제 + +## 2023-07-04 + +- v23.7.2 +- `mediapipe_face_mesh_eyes_only` 모델 추가: `mediapipe_face_mesh`로 감지한 뒤 눈만 사용함. +- 매 배치 시작 전에 `scripts.postprocess`를, 후에 `scripts.process`를 호출함. + - 컨트롤넷을 사용하면 소요 시간이 조금 늘어나지만 몇몇 문제 해결에 도움이 됨. +- `lora_block_weight`를 스크립트 화이트리스트에 추가함. + - 한번이라도 ADetailer를 사용한 사람은 수동으로 추가해야함. + +## 2023-07-03 + +- v23.7.1 +- `process_images`를 진행한 뒤 `StableDiffusionProcessing` 오브젝트의 close를 호출함 +- api 호출로 사용했는지 확인하는 속성 추가 +- `NansException`이 발생했을 때 중지하지 않고 남은 과정 계속 진행함 + +## 2023-07-02 + +- v23.7.0 +- `NansException`이 발생하면 로그에 표시하고 원본 이미지를 반환하게 설정 +- `rich`를 사용한 에러 트레이싱 + - install.py에 `rich` 추가 +- 생성 중에 컴포넌트의 값을 변경하면 args의 값도 함께 변경되는 문제 수정 (issue #180) +- 터미널 로그로 ad_prompt와 ad_negative_prompt에 적용된 실제 프롬프트 확인할 수 있음 (입력과 다를 경우에만) + +## 2023-06-28 + +- v23.6.4 +- 최대 모델 수 5 -> 10개 +- ad_prompt와 ad_negative_prompt에 빈칸으로 놔두면 입력 프롬프트가 사용된다는 문구 추가 +- huggingface 모델 다운로드 실패시 로깅 +- 1st 모델이 `None`일 경우 나머지 입력을 무시하던 문제 수정 +- `--use-cpu` 에 `adetailer` 입력 시 cpu로 yolo모델을 사용함 + +## 2023-06-20 + +- v23.6.3 +- 컨트롤넷 inpaint 모델에 대해, 3가지 모듈을 사용할 수 있도록 함 +- Noise Multiplier 옵션 추가 (PR #149) +- pydantic 최소 버전 1.10.8로 설정 (Issue #146) + +## 2023-06-05 + +- v23.6.2 +- xyz_grid에서 ADetailer를 사용할 수 있게함. + - 8가지 옵션만 1st 탭에 적용되도록 함. + +## 2023-06-01 + +- v23.6.1 +- `inpaint, scribble, lineart, openpose, tile` 5가지 컨트롤넷 모델 지원 (PR #107) +- controlnet guidance start, end 인자 추가 (PR #107) +- `modules.extensions`를 사용하여 컨트롤넷 확장을 불러오고 경로를 알아내로록 변경 +- ui에서 컨트롤넷을 별도 함수로 분리 + +## 2023-05-30 + +- v23.6.0 +- 스크립트의 이름을 `After Detailer`에서 `ADetailer`로 변경 + - API 사용자는 변경 필요함 +- 몇몇 설정 변경 + - `ad_conf` → `ad_confidence`. 0~100 사이의 int → 0.0~1.0 사이의 float + - `ad_inpaint_full_res` → `ad_inpaint_only_masked` + - `ad_inpaint_full_res_padding` → `ad_inpaint_only_masked_padding` +- mediapipe face mesh 모델 추가 + - mediapipe 최소 버전 `0.10.0` + +- rich traceback 제거함 +- huggingface 다운로드 실패할 때 에러가 나지 않게 하고 해당 모델을 제거함 + +## 2023-05-26 + +- v23.5.19 +- 1번째 탭에도 `None` 옵션을 추가함 +- api로 ad controlnet model에 inpaint가 아닌 다른 컨트롤넷 모델을 사용하지 못하도록 막음 +- adetailer 진행중에 total tqdm 진행바 업데이트를 멈춤 +- state.inturrupted 상태에서 adetailer 과정을 중지함 +- 컨트롤넷 process를 각 batch가 끝난 순간에만 호출하도록 변경 + +### 2023-05-25 + +- v23.5.18 +- 컨트롤넷 관련 수정 + - unit의 `input_mode`를 `SIMPLE`로 모두 변경 + - 컨트롤넷 유넷 훅과 하이잭 함수들을 adetailer를 실행할 때에만 되돌리는 기능 추가 + - adetailer 처리가 끝난 뒤 컨트롤넷 스크립트의 process를 다시 진행함. (batch count 2 이상일때의 문제 해결) +- 기본 활성 스크립트 목록에서 컨트롤넷을 뺌 + +### 2023-05-22 + +- v23.5.17 +- 컨트롤넷 확장이 있으면 컨트롤넷 스크립트를 활성화함. (컨트롤넷 관련 문제 해결) +- 모든 컴포넌트에 elem_id 설정 +- ui에 버전을 표시함 + + +### 2023-05-19 + +- v23.5.16 +- 추가한 옵션 + - Mask min/max ratio + - Mask merge mode + - Restore faces after ADetailer +- 옵션들을 Accordion으로 묶음 + +### 2023-05-18 + +- v23.5.15 +- 필요한 것만 임포트하도록 변경 (vae 로딩 오류 없어짐. 로딩 속도 빨라짐) + +### 2023-05-17 + +- v23.5.14 +- `[SKIP]`으로 ad prompt 일부를 건너뛰는 기능 추가 +- bbox 정렬 옵션 추가 +- sd_webui 타입힌트를 만들어냄 +- enable checker와 관련된 api 오류 수정? + +### 2023-05-15 + +- v23.5.13 +- `[SEP]`으로 ad prompt를 분리하여 적용하는 기능 추가 +- enable checker를 다시 pydantic으로 변경함 +- ui 관련 함수를 adetailer.ui 폴더로 분리함 +- controlnet을 사용할 때 모든 controlnet unit 비활성화 +- adetailer 폴더가 없으면 만들게 함 + +### 2023-05-13 + +- v23.5.12 +- `ad_enable`을 제외한 입력이 dict타입으로 들어오도록 변경 + - web api로 사용할 때에 특히 사용하기 쉬움 + - web api breaking change +- `mask_preprocess` 인자를 넣지 않았던 오류 수정 (PR #47) +- huggingface에서 모델을 다운로드하지 않는 옵션 추가 `--ad-no-huggingface` + +### 2023-05-12 + +- v23.5.11 +- `ultralytics` 알람 제거 +- 필요없는 exif 인자 더 제거함 +- `use separate steps` 옵션 추가 +- ui 배치를 조정함 + +### 2023-05-09 + +- v23.5.10 +- 선택한 스크립트만 ADetailer에 적용하는 옵션 추가, 기본값 `True`. 설정 탭에서 지정가능. + - 기본값: `dynamic_prompting,dynamic_thresholding,wildcards,wildcard_recursive` +- `person_yolov8s-seg.pt` 모델 추가 +- `ultralytics`의 최소 버전을 `8.0.97`로 설정 (C:\\ 문제 해결된 버전) + +### 2023-05-08 + +- v23.5.9 +- 2가지 이상의 모델을 사용할 수 있음. 기본값: 2, 최대: 5 +- segment 모델을 사용할 수 있게 함. `person_yolov8n-seg.pt` 추가 + +### 2023-05-07 + +- v23.5.8 +- 프롬프트와 네거티브 프롬프트에 방향키 지원 (PR #24) +- `mask_preprocess`를 추가함. 이전 버전과 시드값이 달라질 가능성 있음! +- 이미지 처리가 일어났을 때에만 before이미지를 저장함 +- 설정창의 레이블을 ADetailer 대신 더 적절하게 수정함 + +### 2023-05-06 + +- v23.5.7 +- `ad_use_cfg_scale` 옵션 추가. cfg 스케일을 따로 사용할지 말지 결정함. +- `ad_enable` 기본값을 `True`에서 `False`로 변경 +- `ad_model`의 기본값을 `None`에서 첫번째 모델로 변경 +- 최소 2개의 입력(ad_enable, ad_model)만 들어오면 작동하게 변경. + +- v23.5.7.post0 +- `init_controlnet_ext`을 controlnet_exists == True일때에만 실행 +- webui를 C드라이브 바로 밑에 설치한 사람들에게 `ultralytics` 경고 표시 + +### 2023-05-05 (어린이날) + +- v23.5.5 +- `Save images before ADetailer` 옵션 추가 +- 입력으로 들어온 인자와 ALL_ARGS의 길이가 다르면 에러메세지 +- README.md에 설치방법 추가 + +- v23.5.6 +- get_args에서 IndexError가 발생하면 자세한 에러메세지를 볼 수 있음 +- AdetailerArgs에 extra_params 내장 +- scripts_args를 딥카피함 +- postprocess_image를 약간 분리함 + +- v23.5.6.post0 +- `init_controlnet_ext`에서 에러메세지를 자세히 볼 수 있음 + +### 2023-05-04 + +- v23.5.4 +- use pydantic for arguments validation +- revert: ad_model to `None` as default +- revert: `__future__` imports +- lazily import yolo and mediapipe + +### 2023-05-03 + +- v23.5.3.post0 +- remove `__future__` imports +- change to copy scripts and scripts args + +- v23.5.3.post1 +- change default ad_model from `None` + +### 2023-05-02 + +- v23.5.3 +- Remove `None` from model list and add `Enable ADetailer` checkbox. +- install.py `skip_install` fix. diff --git a/extensions/adetailer/LICENSE.md b/extensions/adetailer/LICENSE.md new file mode 100644 index 0000000000000000000000000000000000000000..15bc112be2418653138c879e8f15c7b001229324 --- /dev/null +++ b/extensions/adetailer/LICENSE.md @@ -0,0 +1,662 @@ + + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published + by the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. diff --git a/extensions/adetailer/README.md b/extensions/adetailer/README.md new file mode 100644 index 0000000000000000000000000000000000000000..adeced8b6edf25d75bef3d1e3028d94ba7e4f73e --- /dev/null +++ b/extensions/adetailer/README.md @@ -0,0 +1,105 @@ +# !After Detailer + +!After Detailer is a extension for stable diffusion webui, similar to Detection Detailer, except it uses ultralytics instead of the mmdet. + +## Install + +(from Mikubill/sd-webui-controlnet) + +1. Open "Extensions" tab. +2. Open "Install from URL" tab in the tab. +3. Enter `https://github.com/Bing-su/adetailer.git` to "URL for extension's git repository". +4. Press "Install" button. +5. Wait 5 seconds, and you will see the message "Installed into stable-diffusion-webui\extensions\adetailer. Use Installed tab to restart". +6. Go to "Installed" tab, click "Check for updates", and then click "Apply and restart UI". (The next time you can also use this method to update extensions.) +7. Completely restart A1111 webui including your terminal. (If you do not know what is a "terminal", you can reboot your computer: turn your computer off and turn it on again.) + +You can now install it directly from the Extensions tab. + +![image](https://i.imgur.com/g6GdRBT.png) + +You **DON'T** need to download any model from huggingface. + +## Options + +| Model, Prompts | | | +| --------------------------------- | ------------------------------------- | ------------------------------------------------- | +| ADetailer model | Determine what to detect. | `None` = disable | +| ADetailer prompt, negative prompt | Prompts and negative prompts to apply | If left blank, it will use the same as the input. | + +| Detection | | | +| ------------------------------------ | -------------------------------------------------------------------------------------------- | --- | +| Detection model confidence threshold | Only objects with a detection model confidence above this threshold are used for inpainting. | | +| Mask min/max ratio | Only use masks whose area is between those ratios for the area of the entire image. | | + +If you want to exclude objects in the background, try setting the min ratio to around `0.01`. + +| Mask Preprocessing | | | +| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | +| Mask x, y offset | Moves the mask horizontally and vertically by | | +| Mask erosion (-) / dilation (+) | Enlarge or reduce the detected mask. | [opencv example](https://docs.opencv.org/4.7.0/db/df6/tutorial_erosion_dilatation.html) | +| Mask merge mode | `None`: Inpaint each mask
`Merge`: Merge all masks and inpaint
`Merge and Invert`: Merge all masks and Invert, then inpaint | | + +Applied in this order: x, y offset → erosion/dilation → merge/invert. + +#### Inpainting + +![image](https://i.imgur.com/wyWlT1n.png) + +Each option corresponds to a corresponding option on the inpaint tab. + +## ControlNet Inpainting + +You can use the ControlNet extension if you have ControlNet installed and ControlNet models. + +Support `inpaint, scribble, lineart, openpose, tile` controlnet models. Once you choose a model, the preprocessor is set automatically. + +## Model + +| Model | Target | mAP 50 | mAP 50-95 | +| --------------------- | --------------------- | ----------------------------- | ----------------------------- | +| face_yolov8n.pt | 2D / realistic face | 0.660 | 0.366 | +| face_yolov8s.pt | 2D / realistic face | 0.713 | 0.404 | +| hand_yolov8n.pt | 2D / realistic hand | 0.767 | 0.505 | +| person_yolov8n-seg.pt | 2D / realistic person | 0.782 (bbox)
0.761 (mask) | 0.555 (bbox)
0.460 (mask) | +| person_yolov8s-seg.pt | 2D / realistic person | 0.824 (bbox)
0.809 (mask) | 0.605 (bbox)
0.508 (mask) | +| mediapipe_face_full | realistic face | - | - | +| mediapipe_face_short | realistic face | - | - | +| mediapipe_face_mesh | realistic face | - | - | + +The yolo models can be found on huggingface [Bingsu/adetailer](https://huggingface.co/Bingsu/adetailer). + +### User Model + +Put your [ultralytics](https://github.com/ultralytics/ultralytics) model in `webui/models/adetailer`. The model name should end with `.pt` or `.pth`. + +It must be a bbox detection or segment model and use all label. + +### Dataset + +Datasets used for training the yolo models are: + +#### Face + +- [Anime Face CreateML](https://universe.roboflow.com/my-workspace-mph8o/anime-face-createml) +- [xml2txt](https://universe.roboflow.com/0oooooo0/xml2txt-njqx1) +- [AN](https://universe.roboflow.com/sed-b8vkf/an-lfg5i) +- [wider face](http://shuoyang1213.me/WIDERFACE/index.html) + +#### Hand + +- [AnHDet](https://universe.roboflow.com/1-yshhi/anhdet) +- [hand-detection-fuao9](https://universe.roboflow.com/catwithawand/hand-detection-fuao9) + +#### Person + +- [coco2017](https://cocodataset.org/#home) (only person) +- [AniSeg](https://github.com/jerryli27/AniSeg) +- [skytnt/anime-segmentation](https://huggingface.co/datasets/skytnt/anime-segmentation) + +## Example + +![image](https://i.imgur.com/38RSxSO.png) +![image](https://i.imgur.com/2CYgjLx.png) + +[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/F1F1L7V2N) diff --git a/extensions/adetailer/__pycache__/preload.cpython-310.pyc b/extensions/adetailer/__pycache__/preload.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..77087886d397bc0c84f7aaf3fe742ca3bb46d61e Binary files /dev/null and b/extensions/adetailer/__pycache__/preload.cpython-310.pyc differ diff --git a/extensions/adetailer/__pycache__/preload.cpython-311.pyc b/extensions/adetailer/__pycache__/preload.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e6984e45d6e6fda39cdd7b64a8471d9e5c84394a Binary files /dev/null and b/extensions/adetailer/__pycache__/preload.cpython-311.pyc differ diff --git a/extensions/adetailer/adetailer/__init__.py b/extensions/adetailer/adetailer/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..dae6181e4042e320e7854837ed8e96b1da583c83 --- /dev/null +++ b/extensions/adetailer/adetailer/__init__.py @@ -0,0 +1,20 @@ +from .__version__ import __version__ +from .args import AD_ENABLE, ALL_ARGS, ADetailerArgs, EnableChecker +from .common import PredictOutput, get_models +from .mediapipe import mediapipe_predict +from .ultralytics import ultralytics_predict + +AFTER_DETAILER = "ADetailer" + +__all__ = [ + "__version__", + "AD_ENABLE", + "ADetailerArgs", + "AFTER_DETAILER", + "ALL_ARGS", + "EnableChecker", + "PredictOutput", + "get_models", + "mediapipe_predict", + "ultralytics_predict", +] diff --git a/extensions/adetailer/adetailer/__pycache__/__init__.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1e50b4aa5ae5cc18135bea818d7ce18c8639f591 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/__init__.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/__version__.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/__version__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..fe9d047667639c3b20dc710f43530971f4907937 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/__version__.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/args.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/args.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..03e2c9c3f00c1fa86c6a44cdb67b0ba609f7fd76 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/args.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/common.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/common.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..94d6a30569095f9d1ce441530277e5a85eb594b8 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/common.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/mask.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/mask.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..9b171a253e389ef05f39a750f60321f7b6f092ce Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/mask.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/mediapipe.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/mediapipe.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..753084aa5a02f7f12b309378372fac58febd1028 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/mediapipe.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/traceback.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/traceback.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..60c52cd4471fa946c6abf846f05db584de023878 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/traceback.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/ui.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/ui.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1f326948ac768e6fa2f6cbb666b42f8a4f2f82e2 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/ui.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__pycache__/ultralytics.cpython-310.pyc b/extensions/adetailer/adetailer/__pycache__/ultralytics.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5d4eb7ade526178bc02477a2091b2570787c9852 Binary files /dev/null and b/extensions/adetailer/adetailer/__pycache__/ultralytics.cpython-310.pyc differ diff --git a/extensions/adetailer/adetailer/__version__.py b/extensions/adetailer/adetailer/__version__.py new file mode 100644 index 0000000000000000000000000000000000000000..8a190abc29f7440898bc2cd46c63aba14b8af51c --- /dev/null +++ b/extensions/adetailer/adetailer/__version__.py @@ -0,0 +1 @@ +__version__ = "23.7.5" diff --git a/extensions/adetailer/adetailer/args.py b/extensions/adetailer/adetailer/args.py new file mode 100644 index 0000000000000000000000000000000000000000..e7d4178af0e9272cb6e35138ab6773a083b7b871 --- /dev/null +++ b/extensions/adetailer/adetailer/args.py @@ -0,0 +1,214 @@ +from __future__ import annotations + +from collections import UserList +from functools import cached_property, partial +from typing import Any, Literal, NamedTuple, Optional, Union + +import pydantic +from pydantic import ( + BaseModel, + Extra, + NonNegativeFloat, + NonNegativeInt, + PositiveInt, + confloat, + constr, + root_validator, + validator, +) + +cn_model_regex = r".*(inpaint|tile|scribble|lineart|openpose).*|^None$" + + +class Arg(NamedTuple): + attr: str + name: str + + +class ArgsList(UserList): + @cached_property + def attrs(self) -> tuple[str]: + return tuple(attr for attr, _ in self) + + @cached_property + def names(self) -> tuple[str]: + return tuple(name for _, name in self) + + +class ADetailerArgs(BaseModel, extra=Extra.forbid): + ad_model: str = "None" + ad_prompt: str = "" + ad_negative_prompt: str = "" + ad_confidence: confloat(ge=0.0, le=1.0) = 0.3 + ad_mask_min_ratio: confloat(ge=0.0, le=1.0) = 0.0 + ad_mask_max_ratio: confloat(ge=0.0, le=1.0) = 1.0 + ad_dilate_erode: int = 4 + ad_x_offset: int = 0 + ad_y_offset: int = 0 + ad_mask_merge_invert: Literal["None", "Merge", "Merge and Invert"] = "None" + ad_mask_blur: NonNegativeInt = 4 + ad_denoising_strength: confloat(ge=0.0, le=1.0) = 0.4 + ad_inpaint_only_masked: bool = True + ad_inpaint_only_masked_padding: NonNegativeInt = 32 + ad_use_inpaint_width_height: bool = False + ad_inpaint_width: PositiveInt = 512 + ad_inpaint_height: PositiveInt = 512 + ad_use_steps: bool = False + ad_steps: PositiveInt = 28 + ad_use_cfg_scale: bool = False + ad_cfg_scale: NonNegativeFloat = 7.0 + ad_use_noise_multiplier: bool = False + ad_noise_multiplier: confloat(ge=0.5, le=1.5) = 1.0 + ad_restore_face: bool = False + ad_controlnet_model: constr(regex=cn_model_regex) = "None" + ad_controlnet_module: Optional[constr(regex=r".*inpaint.*|^None$")] = None + ad_controlnet_weight: confloat(ge=0.0, le=1.0) = 1.0 + ad_controlnet_guidance_start: confloat(ge=0.0, le=1.0) = 0.0 + ad_controlnet_guidance_end: confloat(ge=0.0, le=1.0) = 1.0 + is_api: bool = True + + @root_validator(skip_on_failure=True) + def ad_controlnt_module_validator(cls, values): # noqa: N805 + cn_model = values.get("ad_controlnet_model", "None") + cn_module = values.get("ad_controlnet_module", None) + if "inpaint" not in cn_model or cn_module == "None": + values["ad_controlnet_module"] = None + return values + + @validator("is_api", pre=True) + def is_api_validator(cls, v: Any): # noqa: N805 + "tuple is json serializable but cannot be made with json deserialize." + return type(v) is not tuple + + @staticmethod + def ppop( + p: dict[str, Any], + key: str, + pops: list[str] | None = None, + cond: Any = None, + ) -> None: + if pops is None: + pops = [key] + if key not in p: + return + value = p[key] + cond = (not bool(value)) if cond is None else value == cond + + if cond: + for k in pops: + p.pop(k, None) + + def extra_params(self, suffix: str = "") -> dict[str, Any]: + if self.ad_model == "None": + return {} + + p = {name: getattr(self, attr) for attr, name in ALL_ARGS} + ppop = partial(self.ppop, p) + + ppop("ADetailer prompt") + ppop("ADetailer negative prompt") + ppop("ADetailer mask min ratio", cond=0.0) + ppop("ADetailer mask max ratio", cond=1.0) + ppop("ADetailer x offset", cond=0) + ppop("ADetailer y offset", cond=0) + ppop("ADetailer mask merge/invert", cond="None") + ppop("ADetailer inpaint only masked", ["ADetailer inpaint padding"]) + ppop( + "ADetailer use inpaint width/height", + [ + "ADetailer use inpaint width/height", + "ADetailer inpaint width", + "ADetailer inpaint height", + ], + ) + ppop( + "ADetailer use separate steps", + ["ADetailer use separate steps", "ADetailer steps"], + ) + ppop( + "ADetailer use separate CFG scale", + ["ADetailer use separate CFG scale", "ADetailer CFG scale"], + ) + ppop( + "ADetailer use separate noise multiplier", + ["ADetailer use separate noise multiplier", "ADetailer noise multiplier"], + ) + + ppop("ADetailer restore face") + ppop( + "ADetailer ControlNet model", + [ + "ADetailer ControlNet model", + "ADetailer ControlNet module", + "ADetailer ControlNet weight", + "ADetailer ControlNet guidance start", + "ADetailer ControlNet guidance end", + ], + cond="None", + ) + ppop("ADetailer ControlNet module") + ppop("ADetailer ControlNet weight", cond=1.0) + ppop("ADetailer ControlNet guidance start", cond=0.0) + ppop("ADetailer ControlNet guidance end", cond=1.0) + + if suffix: + p = {k + suffix: v for k, v in p.items()} + + return p + + +class EnableChecker(BaseModel): + enable: bool + arg_list: list + + def is_enabled(self) -> bool: + ad_model = ALL_ARGS[0].attr + if not self.enable: + return False + return any(arg.get(ad_model, "None") != "None" for arg in self.arg_list) + + +_all_args = [ + ("ad_enable", "ADetailer enable"), + ("ad_model", "ADetailer model"), + ("ad_prompt", "ADetailer prompt"), + ("ad_negative_prompt", "ADetailer negative prompt"), + ("ad_confidence", "ADetailer confidence"), + ("ad_mask_min_ratio", "ADetailer mask min ratio"), + ("ad_mask_max_ratio", "ADetailer mask max ratio"), + ("ad_x_offset", "ADetailer x offset"), + ("ad_y_offset", "ADetailer y offset"), + ("ad_dilate_erode", "ADetailer dilate/erode"), + ("ad_mask_merge_invert", "ADetailer mask merge/invert"), + ("ad_mask_blur", "ADetailer mask blur"), + ("ad_denoising_strength", "ADetailer denoising strength"), + ("ad_inpaint_only_masked", "ADetailer inpaint only masked"), + ("ad_inpaint_only_masked_padding", "ADetailer inpaint padding"), + ("ad_use_inpaint_width_height", "ADetailer use inpaint width/height"), + ("ad_inpaint_width", "ADetailer inpaint width"), + ("ad_inpaint_height", "ADetailer inpaint height"), + ("ad_use_steps", "ADetailer use separate steps"), + ("ad_steps", "ADetailer steps"), + ("ad_use_cfg_scale", "ADetailer use separate CFG scale"), + ("ad_cfg_scale", "ADetailer CFG scale"), + ("ad_use_noise_multiplier", "ADetailer use separate noise multiplier"), + ("ad_noise_multiplier", "ADetailer noise multiplier"), + ("ad_restore_face", "ADetailer restore face"), + ("ad_controlnet_model", "ADetailer ControlNet model"), + ("ad_controlnet_module", "ADetailer ControlNet module"), + ("ad_controlnet_weight", "ADetailer ControlNet weight"), + ("ad_controlnet_guidance_start", "ADetailer ControlNet guidance start"), + ("ad_controlnet_guidance_end", "ADetailer ControlNet guidance end"), +] + +AD_ENABLE = Arg(*_all_args[0]) +_args = [Arg(*args) for args in _all_args[1:]] +ALL_ARGS = ArgsList(_args) + +BBOX_SORTBY = [ + "None", + "Position (left to right)", + "Position (center to edge)", + "Area (large to small)", +] +MASK_MERGE_INVERT = ["None", "Merge", "Merge and Invert"] diff --git a/extensions/adetailer/adetailer/common.py b/extensions/adetailer/adetailer/common.py new file mode 100644 index 0000000000000000000000000000000000000000..a29bdd5beb2ac3e1f027398dc176e70fb77d082d --- /dev/null +++ b/extensions/adetailer/adetailer/common.py @@ -0,0 +1,127 @@ +from __future__ import annotations + +from collections import OrderedDict +from dataclasses import dataclass, field +from pathlib import Path +from typing import Optional, Union + +from huggingface_hub import hf_hub_download +from PIL import Image, ImageDraw +from rich import print + +repo_id = "Bingsu/adetailer" + + +@dataclass +class PredictOutput: + bboxes: list[list[int | float]] = field(default_factory=list) + masks: list[Image.Image] = field(default_factory=list) + preview: Optional[Image.Image] = None + + +def hf_download(file: str): + try: + path = hf_hub_download(repo_id, file) + except Exception: + msg = f"[-] ADetailer: Failed to load model {file!r} from huggingface" + print(msg) + path = "INVALID" + return path + + +def get_models( + model_dir: Union[str, Path], huggingface: bool = True +) -> OrderedDict[str, Optional[str]]: + model_dir = Path(model_dir) + if model_dir.is_dir(): + model_paths = [ + p + for p in model_dir.rglob("*") + if p.is_file() and p.suffix in (".pt", ".pth") + ] + else: + model_paths = [] + + models = OrderedDict() + if huggingface: + models.update( + { + "face_yolov8n.pt": hf_download("face_yolov8n.pt"), + "face_yolov8s.pt": hf_download("face_yolov8s.pt"), + "hand_yolov8n.pt": hf_download("hand_yolov8n.pt"), + "person_yolov8n-seg.pt": hf_download("person_yolov8n-seg.pt"), + "person_yolov8s-seg.pt": hf_download("person_yolov8s-seg.pt"), + } + ) + models.update( + { + "mediapipe_face_full": None, + "mediapipe_face_short": None, + "mediapipe_face_mesh": None, + "mediapipe_face_mesh_eyes_only": None, + } + ) + + invalid_keys = [k for k, v in models.items() if v == "INVALID"] + for key in invalid_keys: + models.pop(key) + + for path in model_paths: + if path.name in models: + continue + models[path.name] = str(path) + + return models + + +def create_mask_from_bbox( + bboxes: list[list[float]], shape: tuple[int, int] +) -> list[Image.Image]: + """ + Parameters + ---------- + bboxes: list[list[float]] + list of [x1, y1, x2, y2] + bounding boxes + shape: tuple[int, int] + shape of the image (width, height) + + Returns + ------- + masks: list[Image.Image] + A list of masks + + """ + masks = [] + for bbox in bboxes: + mask = Image.new("L", shape, 0) + mask_draw = ImageDraw.Draw(mask) + mask_draw.rectangle(bbox, fill=255) + masks.append(mask) + return masks + + +def create_bbox_from_mask( + masks: list[Image.Image], shape: tuple[int, int] +) -> list[list[int]]: + """ + Parameters + ---------- + masks: list[Image.Image] + A list of masks + shape: tuple[int, int] + shape of the image (width, height) + + Returns + ------- + bboxes: list[list[float]] + A list of bounding boxes + + """ + bboxes = [] + for mask in masks: + mask = mask.resize(shape) + bbox = mask.getbbox() + if bbox is not None: + bboxes.append(list(bbox)) + return bboxes diff --git a/extensions/adetailer/adetailer/mask.py b/extensions/adetailer/adetailer/mask.py new file mode 100644 index 0000000000000000000000000000000000000000..9209b456fac2d7fda987c0b076874e3bb2cd7ee8 --- /dev/null +++ b/extensions/adetailer/adetailer/mask.py @@ -0,0 +1,245 @@ +from __future__ import annotations + +from enum import IntEnum +from functools import partial, reduce +from math import dist + +import cv2 +import numpy as np +from PIL import Image, ImageChops + +from adetailer.args import MASK_MERGE_INVERT +from adetailer.common import PredictOutput + + +class SortBy(IntEnum): + NONE = 0 + LEFT_TO_RIGHT = 1 + CENTER_TO_EDGE = 2 + AREA = 3 + + +class MergeInvert(IntEnum): + NONE = 0 + MERGE = 1 + MERGE_INVERT = 2 + + +def _dilate(arr: np.ndarray, value: int) -> np.ndarray: + kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (value, value)) + return cv2.dilate(arr, kernel, iterations=1) + + +def _erode(arr: np.ndarray, value: int) -> np.ndarray: + kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (value, value)) + return cv2.erode(arr, kernel, iterations=1) + + +def dilate_erode(img: Image.Image, value: int) -> Image.Image: + """ + The dilate_erode function takes an image and a value. + If the value is positive, it dilates the image by that amount. + If the value is negative, it erodes the image by that amount. + + Parameters + ---------- + img: PIL.Image.Image + the image to be processed + value: int + kernel size of dilation or erosion + + Returns + ------- + PIL.Image.Image + The image that has been dilated or eroded + """ + if value == 0: + return img + + arr = np.array(img) + arr = _dilate(arr, value) if value > 0 else _erode(arr, -value) + + return Image.fromarray(arr) + + +def offset(img: Image.Image, x: int = 0, y: int = 0) -> Image.Image: + """ + The offset function takes an image and offsets it by a given x(→) and y(↑) value. + + Parameters + ---------- + mask: Image.Image + Pass the mask image to the function + x: int + → + y: int + ↑ + + Returns + ------- + PIL.Image.Image + A new image that is offset by x and y + """ + return ImageChops.offset(img, x, -y) + + +def is_all_black(img: Image.Image) -> bool: + arr = np.array(img) + return cv2.countNonZero(arr) == 0 + + +def bbox_area(bbox: list[float]): + return (bbox[2] - bbox[0]) * (bbox[3] - bbox[1]) + + +def mask_preprocess( + masks: list[Image.Image], + kernel: int = 0, + x_offset: int = 0, + y_offset: int = 0, + merge_invert: int | MergeInvert | str = MergeInvert.NONE, +) -> list[Image.Image]: + """ + The mask_preprocess function takes a list of masks and preprocesses them. + It dilates and erodes the masks, and offsets them by x_offset and y_offset. + + Parameters + ---------- + masks: list[Image.Image] + A list of masks + kernel: int + kernel size of dilation or erosion + x_offset: int + → + y_offset: int + ↑ + + Returns + ------- + list[Image.Image] + A list of processed masks + """ + if not masks: + return [] + + if x_offset != 0 or y_offset != 0: + masks = [offset(m, x_offset, y_offset) for m in masks] + + if kernel != 0: + masks = [dilate_erode(m, kernel) for m in masks] + masks = [m for m in masks if not is_all_black(m)] + + return mask_merge_invert(masks, mode=merge_invert) + + +# Bbox sorting +def _key_left_to_right(bbox: list[float]) -> float: + """ + Left to right + + Parameters + ---------- + bbox: list[float] + list of [x1, y1, x2, y2] + """ + return bbox[0] + + +def _key_center_to_edge(bbox: list[float], *, center: tuple[float, float]) -> float: + """ + Center to edge + + Parameters + ---------- + bbox: list[float] + list of [x1, y1, x2, y2] + image: Image.Image + the image + """ + bbox_center = ((bbox[0] + bbox[2]) / 2, (bbox[1] + bbox[3]) / 2) + return dist(center, bbox_center) + + +def _key_area(bbox: list[float]) -> float: + """ + Large to small + + Parameters + ---------- + bbox: list[float] + list of [x1, y1, x2, y2] + """ + return -bbox_area(bbox) + + +def sort_bboxes( + pred: PredictOutput, order: int | SortBy = SortBy.NONE +) -> PredictOutput: + if order == SortBy.NONE or len(pred.bboxes) <= 1: + return pred + + if order == SortBy.LEFT_TO_RIGHT: + key = _key_left_to_right + elif order == SortBy.CENTER_TO_EDGE: + width, height = pred.preview.size + center = (width / 2, height / 2) + key = partial(_key_center_to_edge, center=center) + elif order == SortBy.AREA: + key = _key_area + else: + raise RuntimeError + + items = len(pred.bboxes) + idx = sorted(range(items), key=lambda i: key(pred.bboxes[i])) + pred.bboxes = [pred.bboxes[i] for i in idx] + pred.masks = [pred.masks[i] for i in idx] + return pred + + +# Filter by ratio +def is_in_ratio(bbox: list[float], low: float, high: float, orig_area: int) -> bool: + area = bbox_area(bbox) + return low <= area / orig_area <= high + + +def filter_by_ratio(pred: PredictOutput, low: float, high: float) -> PredictOutput: + if not pred.bboxes: + return pred + + w, h = pred.preview.size + orig_area = w * h + items = len(pred.bboxes) + idx = [i for i in range(items) if is_in_ratio(pred.bboxes[i], low, high, orig_area)] + pred.bboxes = [pred.bboxes[i] for i in idx] + pred.masks = [pred.masks[i] for i in idx] + return pred + + +# Merge / Invert +def mask_merge(masks: list[Image.Image]) -> list[Image.Image]: + arrs = [np.array(m) for m in masks] + arr = reduce(cv2.bitwise_or, arrs) + return [Image.fromarray(arr)] + + +def mask_invert(masks: list[Image.Image]) -> list[Image.Image]: + return [ImageChops.invert(m) for m in masks] + + +def mask_merge_invert( + masks: list[Image.Image], mode: int | MergeInvert | str +) -> list[Image.Image]: + if isinstance(mode, str): + mode = MASK_MERGE_INVERT.index(mode) + + if mode == MergeInvert.NONE or not masks: + return masks + + if mode == MergeInvert.MERGE: + return mask_merge(masks) + + if mode == MergeInvert.MERGE_INVERT: + merged = mask_merge(masks) + return mask_invert(merged) + + raise RuntimeError diff --git a/extensions/adetailer/adetailer/mediapipe.py b/extensions/adetailer/adetailer/mediapipe.py new file mode 100644 index 0000000000000000000000000000000000000000..17fb2ccad1425430a994733be33cb2f0463bcb56 --- /dev/null +++ b/extensions/adetailer/adetailer/mediapipe.py @@ -0,0 +1,184 @@ +from __future__ import annotations + +from functools import partial + +import numpy as np +from PIL import Image, ImageDraw + +from adetailer import PredictOutput +from adetailer.common import create_bbox_from_mask, create_mask_from_bbox + + +def mediapipe_predict( + model_type: str, image: Image.Image, confidence: float = 0.3 +) -> PredictOutput: + mapping = { + "mediapipe_face_short": partial(mediapipe_face_detection, 0), + "mediapipe_face_full": partial(mediapipe_face_detection, 1), + "mediapipe_face_mesh": mediapipe_face_mesh, + "mediapipe_face_mesh_eyes_only": mediapipe_face_mesh_eyes_only, + } + if model_type in mapping: + func = mapping[model_type] + return func(image, confidence) + msg = f"[-] ADetailer: Invalid mediapipe model type: {model_type}, Available: {list(mapping.keys())!r}" + raise RuntimeError(msg) + + +def mediapipe_face_detection( + model_type: int, image: Image.Image, confidence: float = 0.3 +) -> PredictOutput: + import mediapipe as mp + + img_width, img_height = image.size + + mp_face_detection = mp.solutions.face_detection + draw_util = mp.solutions.drawing_utils + + img_array = np.array(image) + + with mp_face_detection.FaceDetection( + model_selection=model_type, min_detection_confidence=confidence + ) as face_detector: + pred = face_detector.process(img_array) + + if pred.detections is None: + return PredictOutput() + + preview_array = img_array.copy() + + bboxes = [] + for detection in pred.detections: + draw_util.draw_detection(preview_array, detection) + + bbox = detection.location_data.relative_bounding_box + x1 = bbox.xmin * img_width + y1 = bbox.ymin * img_height + w = bbox.width * img_width + h = bbox.height * img_height + x2 = x1 + w + y2 = y1 + h + + bboxes.append([x1, y1, x2, y2]) + + masks = create_mask_from_bbox(bboxes, image.size) + preview = Image.fromarray(preview_array) + + return PredictOutput(bboxes=bboxes, masks=masks, preview=preview) + + +def get_convexhull(points: np.ndarray) -> list[tuple[int, int]]: + """ + Parameters + ---------- + points: An ndarray of shape (n, 2) containing the 2D points. + + Returns + ------- + list[tuple[int, int]]: Input for the draw.polygon function + """ + from scipy.spatial import ConvexHull + + hull = ConvexHull(points) + vertices = hull.vertices + return list(zip(points[vertices, 0], points[vertices, 1])) + + +def mediapipe_face_mesh(image: Image.Image, confidence: float = 0.3) -> PredictOutput: + import mediapipe as mp + + mp_face_mesh = mp.solutions.face_mesh + draw_util = mp.solutions.drawing_utils + drawing_styles = mp.solutions.drawing_styles + + w, h = image.size + + with mp_face_mesh.FaceMesh( + static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence + ) as face_mesh: + arr = np.array(image) + pred = face_mesh.process(arr) + + if pred.multi_face_landmarks is None: + return PredictOutput() + + preview = arr.copy() + masks = [] + + for landmarks in pred.multi_face_landmarks: + draw_util.draw_landmarks( + image=preview, + landmark_list=landmarks, + connections=mp_face_mesh.FACEMESH_TESSELATION, + landmark_drawing_spec=None, + connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(), + ) + + points = np.array([(land.x * w, land.y * h) for land in landmarks.landmark]) + outline = get_convexhull(points) + + mask = Image.new("L", image.size, "black") + draw = ImageDraw.Draw(mask) + draw.polygon(outline, fill="white") + masks.append(mask) + + bboxes = create_bbox_from_mask(masks, image.size) + preview = Image.fromarray(preview) + return PredictOutput(bboxes=bboxes, masks=masks, preview=preview) + + +def mediapipe_face_mesh_eyes_only( + image: Image.Image, confidence: float = 0.3 +) -> PredictOutput: + import mediapipe as mp + + mp_face_mesh = mp.solutions.face_mesh + + left_idx = np.array(list(mp_face_mesh.FACEMESH_LEFT_EYE)).flatten() + right_idx = np.array(list(mp_face_mesh.FACEMESH_RIGHT_EYE)).flatten() + + w, h = image.size + + with mp_face_mesh.FaceMesh( + static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence + ) as face_mesh: + arr = np.array(image) + pred = face_mesh.process(arr) + + if pred.multi_face_landmarks is None: + return PredictOutput() + + preview = image.copy() + masks = [] + + for landmarks in pred.multi_face_landmarks: + points = np.array([(land.x * w, land.y * h) for land in landmarks.landmark]) + left_eyes = points[left_idx] + right_eyes = points[right_idx] + left_outline = get_convexhull(left_eyes) + right_outline = get_convexhull(right_eyes) + + mask = Image.new("L", image.size, "black") + draw = ImageDraw.Draw(mask) + for outline in (left_outline, right_outline): + draw.polygon(outline, fill="white") + masks.append(mask) + + bboxes = create_bbox_from_mask(masks, image.size) + preview = draw_preview(preview, bboxes, masks) + return PredictOutput(bboxes=bboxes, masks=masks, preview=preview) + + +def draw_preview( + preview: Image.Image, bboxes: list[list[int]], masks: list[Image.Image] +) -> Image.Image: + red = Image.new("RGB", preview.size, "red") + for mask in masks: + masked = Image.composite(red, preview, mask) + preview = Image.blend(preview, masked, 0.25) + + draw = ImageDraw.Draw(preview) + for bbox in bboxes: + draw.rectangle(bbox, outline="red", width=2) + + return preview diff --git a/extensions/adetailer/adetailer/traceback.py b/extensions/adetailer/adetailer/traceback.py new file mode 100644 index 0000000000000000000000000000000000000000..03e9afeac789fb9cc5472145e1a716dd244b0460 --- /dev/null +++ b/extensions/adetailer/adetailer/traceback.py @@ -0,0 +1,158 @@ +from __future__ import annotations + +import io +import platform +import sys +from typing import Any, Callable + +from rich.console import Console, Group +from rich.panel import Panel +from rich.table import Table +from rich.traceback import Traceback + +from adetailer.__version__ import __version__ + + +def processing(*args: Any) -> dict[str, Any]: + try: + from modules.processing import ( + StableDiffusionProcessingImg2Img, + StableDiffusionProcessingTxt2Img, + ) + except ImportError: + return {} + + p = None + for arg in args: + if isinstance( + arg, (StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img) + ): + p = arg + break + + if p is None: + return {} + + info = { + "prompt": p.prompt, + "negative_prompt": p.negative_prompt, + "n_iter": p.n_iter, + "batch_size": p.batch_size, + "width": p.width, + "height": p.height, + "sampler_name": p.sampler_name, + "enable_hr": getattr(p, "enable_hr", False), + "hr_upscaler": getattr(p, "hr_upscaler", ""), + } + + info.update(sd_models()) + return info + + +def sd_models() -> dict[str, str]: + try: + from modules import shared + + opts = shared.opts + except Exception: + return {} + + return { + "checkpoint": getattr(opts, "sd_model_checkpoint", "------"), + "vae": getattr(opts, "sd_vae", "------"), + "unet": getattr(opts, "sd_unet", "------"), + } + + +def ad_args(*args: Any) -> dict[str, Any]: + ad_args = [ + arg + for arg in args + if isinstance(arg, dict) and arg.get("ad_model", "None") != "None" + ] + if not ad_args: + return {} + + arg0 = ad_args[0] + is_api = arg0.get("is_api", True) + return { + "version": __version__, + "ad_model": arg0["ad_model"], + "ad_prompt": arg0.get("ad_prompt", ""), + "ad_negative_prompt": arg0.get("ad_negative_prompt", ""), + "ad_controlnet_model": arg0.get("ad_controlnet_model", "None"), + "is_api": type(is_api) is not tuple, + } + + +def sys_info() -> dict[str, Any]: + try: + import launch + + version = launch.git_tag() + commit = launch.commit_hash() + except Exception: + version = commit = "------" + + return { + "Platform": platform.platform(), + "Python": sys.version, + "Version": version, + "Commit": commit, + "Commandline": sys.argv, + } + + +def get_table(title: str, data: dict[str, Any]) -> Table: + table = Table(title=title, highlight=True) + table.add_column(" ", justify="right", style="dim") + table.add_column("Value") + for key, value in data.items(): + if not isinstance(value, str): + value = repr(value) + table.add_row(key, value) + + return table + + +def force_terminal_value(): + try: + from modules.shared import cmd_opts + + return True if hasattr(cmd_opts, "skip_torch_cuda_test") else None + except Exception: + return None + + +def rich_traceback(func: Callable) -> Callable: + force_terminal = force_terminal_value() + + def wrapper(*args, **kwargs): + string = io.StringIO() + width = Console().width + width = width - 4 if width > 4 else None + console = Console(file=string, force_terminal=force_terminal, width=width) + try: + return func(*args, **kwargs) + except Exception as e: + tables = [ + get_table(title, data) + for title, data in [ + ("System info", sys_info()), + ("Inputs", processing(*args)), + ("ADetailer", ad_args(*args)), + ] + if data + ] + tables.append(Traceback()) + + console.print(Panel(Group(*tables))) + output = "\n" + string.getvalue() + + try: + error = e.__class__(output) + except Exception: + error = RuntimeError(output) + raise error from None + + return wrapper diff --git a/extensions/adetailer/adetailer/ui.py b/extensions/adetailer/adetailer/ui.py new file mode 100644 index 0000000000000000000000000000000000000000..876a910eaddb962ec02a0ac88cc3354e9494b6c6 --- /dev/null +++ b/extensions/adetailer/adetailer/ui.py @@ -0,0 +1,505 @@ +from __future__ import annotations + +from functools import partial +from types import SimpleNamespace +from typing import Any + +import gradio as gr + +from adetailer import AFTER_DETAILER, __version__ +from adetailer.args import AD_ENABLE, ALL_ARGS, MASK_MERGE_INVERT +from controlnet_ext import controlnet_exists, get_cn_models + +cn_module_choices = [ + "inpaint_global_harmonious", + "inpaint_only", + "inpaint_only+lama", +] + + +class Widgets(SimpleNamespace): + def tolist(self): + return [getattr(self, attr) for attr in ALL_ARGS.attrs] + + +def gr_interactive(value: bool = True): + return gr.update(interactive=value) + + +def ordinal(n: int) -> str: + d = {1: "st", 2: "nd", 3: "rd"} + return str(n) + ("th" if 11 <= n % 100 <= 13 else d.get(n % 10, "th")) + + +def suffix(n: int, c: str = " ") -> str: + return "" if n == 0 else c + ordinal(n + 1) + + +def on_widget_change(state: dict, value: Any, *, attr: str): + state[attr] = value + return state + + +def on_generate_click(state: dict, *values: Any): + for attr, value in zip(ALL_ARGS.attrs, values): + state[attr] = value + state["is_api"] = () + return state + + +def on_cn_model_update(cn_model: str): + if "inpaint" in cn_model: + return gr.update( + visible=True, choices=cn_module_choices, value=cn_module_choices[0] + ) + return gr.update(visible=False, choices=["None"], value="None") + + +def elem_id(item_id: str, n: int, is_img2img: bool) -> str: + tap = "img2img" if is_img2img else "txt2img" + suf = suffix(n, "_") + return f"script_{tap}_adetailer_{item_id}{suf}" + + +def adui( + num_models: int, + is_img2img: bool, + model_list: list[str], + t2i_button: gr.Button, + i2i_button: gr.Button, +): + states = [] + infotext_fields = [] + eid = partial(elem_id, n=0, is_img2img=is_img2img) + + with gr.Accordion(AFTER_DETAILER, open=False, elem_id=eid("ad_main_accordion")): + with gr.Row(): + with gr.Column(scale=6): + ad_enable = gr.Checkbox( + label="Enable ADetailer", + value=False, + visible=True, + elem_id=eid("ad_enable"), + ) + + with gr.Column(scale=1, min_width=180): + gr.Markdown( + f"v{__version__}", + elem_id=eid("ad_version"), + ) + + infotext_fields.append((ad_enable, AD_ENABLE.name)) + + with gr.Group(), gr.Tabs(): + for n in range(num_models): + with gr.Tab(ordinal(n + 1)): + state, infofields = one_ui_group( + n=n, + is_img2img=is_img2img, + model_list=model_list, + t2i_button=t2i_button, + i2i_button=i2i_button, + ) + + states.append(state) + infotext_fields.extend(infofields) + + # components: [bool, dict, dict, ...] + components = [ad_enable, *states] + return components, infotext_fields + + +def one_ui_group( + n: int, + is_img2img: bool, + model_list: list[str], + t2i_button: gr.Button, + i2i_button: gr.Button, +): + w = Widgets() + state = gr.State({}) + eid = partial(elem_id, n=n, is_img2img=is_img2img) + + with gr.Row(): + model_choices = [*model_list, "None"] if n == 0 else ["None", *model_list] + + w.ad_model = gr.Dropdown( + label="ADetailer model" + suffix(n), + choices=model_choices, + value=model_choices[0], + visible=True, + type="value", + elem_id=eid("ad_model"), + ) + + with gr.Group(): + with gr.Row(elem_id=eid("ad_toprow_prompt")): + w.ad_prompt = gr.Textbox( + label="ad_prompt" + suffix(n), + show_label=False, + lines=3, + placeholder="ADetailer prompt" + + suffix(n) + + "\nIf blank, the main prompt is used.", + elem_id=eid("ad_prompt"), + ) + + with gr.Row(elem_id=eid("ad_toprow_negative_prompt")): + w.ad_negative_prompt = gr.Textbox( + label="ad_negative_prompt" + suffix(n), + show_label=False, + lines=2, + placeholder="ADetailer negative prompt" + + suffix(n) + + "\nIf blank, the main negative prompt is used.", + elem_id=eid("ad_negative_prompt"), + ) + + with gr.Group(): + with gr.Accordion( + "Detection", open=False, elem_id=eid("ad_detection_accordion") + ): + detection(w, n, is_img2img) + + with gr.Accordion( + "Mask Preprocessing", + open=False, + elem_id=eid("ad_mask_preprocessing_accordion"), + ): + mask_preprocessing(w, n, is_img2img) + + with gr.Accordion( + "Inpainting", open=False, elem_id=eid("ad_inpainting_accordion") + ): + inpainting(w, n, is_img2img) + + with gr.Group(): + controlnet(w, n, is_img2img) + + all_inputs = [state, *w.tolist()] + target_button = i2i_button if is_img2img else t2i_button + target_button.click( + fn=on_generate_click, inputs=all_inputs, outputs=state, queue=False + ) + + infotext_fields = [(getattr(w, attr), name + suffix(n)) for attr, name in ALL_ARGS] + + return state, infotext_fields + + +def detection(w: Widgets, n: int, is_img2img: bool): + eid = partial(elem_id, n=n, is_img2img=is_img2img) + + with gr.Row(): + with gr.Column(): + w.ad_confidence = gr.Slider( + label="Detection model confidence threshold" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.01, + value=0.3, + visible=True, + elem_id=eid("ad_confidence"), + ) + + with gr.Column(variant="compact"): + w.ad_mask_min_ratio = gr.Slider( + label="Mask min area ratio" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.001, + value=0.0, + visible=True, + elem_id=eid("ad_mask_min_ratio"), + ) + w.ad_mask_max_ratio = gr.Slider( + label="Mask max area ratio" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.001, + value=1.0, + visible=True, + elem_id=eid("ad_mask_max_ratio"), + ) + + +def mask_preprocessing(w: Widgets, n: int, is_img2img: bool): + eid = partial(elem_id, n=n, is_img2img=is_img2img) + + with gr.Group(): + with gr.Row(): + with gr.Column(variant="compact"): + w.ad_x_offset = gr.Slider( + label="Mask x(→) offset" + suffix(n), + minimum=-200, + maximum=200, + step=1, + value=0, + visible=True, + elem_id=eid("ad_x_offset"), + ) + w.ad_y_offset = gr.Slider( + label="Mask y(↑) offset" + suffix(n), + minimum=-200, + maximum=200, + step=1, + value=0, + visible=True, + elem_id=eid("ad_y_offset"), + ) + + with gr.Column(variant="compact"): + w.ad_dilate_erode = gr.Slider( + label="Mask erosion (-) / dilation (+)" + suffix(n), + minimum=-128, + maximum=128, + step=4, + value=4, + visible=True, + elem_id=eid("ad_dilate_erode"), + ) + + with gr.Row(): + w.ad_mask_merge_invert = gr.Radio( + label="Mask merge mode" + suffix(n), + choices=MASK_MERGE_INVERT, + value="None", + elem_id=eid("ad_mask_merge_invert"), + ) + + +def inpainting(w: Widgets, n: int, is_img2img: bool): + eid = partial(elem_id, n=n, is_img2img=is_img2img) + + with gr.Group(): + with gr.Row(): + w.ad_mask_blur = gr.Slider( + label="Inpaint mask blur" + suffix(n), + minimum=0, + maximum=64, + step=1, + value=4, + visible=True, + elem_id=eid("ad_mask_blur"), + ) + + w.ad_denoising_strength = gr.Slider( + label="Inpaint denoising strength" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.01, + value=0.4, + visible=True, + elem_id=eid("ad_denoising_strength"), + ) + + with gr.Row(): + with gr.Column(variant="compact"): + w.ad_inpaint_only_masked = gr.Checkbox( + label="Inpaint only masked" + suffix(n), + value=True, + visible=True, + elem_id=eid("ad_inpaint_only_masked"), + ) + w.ad_inpaint_only_masked_padding = gr.Slider( + label="Inpaint only masked padding, pixels" + suffix(n), + minimum=0, + maximum=256, + step=4, + value=32, + visible=True, + elem_id=eid("ad_inpaint_only_masked_padding"), + ) + + w.ad_inpaint_only_masked.change( + gr_interactive, + inputs=w.ad_inpaint_only_masked, + outputs=w.ad_inpaint_only_masked_padding, + queue=False, + ) + + with gr.Column(variant="compact"): + w.ad_use_inpaint_width_height = gr.Checkbox( + label="Use separate width/height" + suffix(n), + value=False, + visible=True, + elem_id=eid("ad_use_inpaint_width_height"), + ) + + w.ad_inpaint_width = gr.Slider( + label="inpaint width" + suffix(n), + minimum=64, + maximum=2048, + step=4, + value=512, + visible=True, + elem_id=eid("ad_inpaint_width"), + ) + + w.ad_inpaint_height = gr.Slider( + label="inpaint height" + suffix(n), + minimum=64, + maximum=2048, + step=4, + value=512, + visible=True, + elem_id=eid("ad_inpaint_height"), + ) + + w.ad_use_inpaint_width_height.change( + lambda value: (gr_interactive(value), gr_interactive(value)), + inputs=w.ad_use_inpaint_width_height, + outputs=[w.ad_inpaint_width, w.ad_inpaint_height], + queue=False, + ) + + with gr.Row(): + with gr.Column(variant="compact"): + w.ad_use_steps = gr.Checkbox( + label="Use separate steps" + suffix(n), + value=False, + visible=True, + elem_id=eid("ad_use_steps"), + ) + + w.ad_steps = gr.Slider( + label="ADetailer steps" + suffix(n), + minimum=1, + maximum=150, + step=1, + value=28, + visible=True, + elem_id=eid("ad_steps"), + ) + + w.ad_use_steps.change( + gr_interactive, + inputs=w.ad_use_steps, + outputs=w.ad_steps, + queue=False, + ) + + with gr.Column(variant="compact"): + w.ad_use_cfg_scale = gr.Checkbox( + label="Use separate CFG scale" + suffix(n), + value=False, + visible=True, + elem_id=eid("ad_use_cfg_scale"), + ) + + w.ad_cfg_scale = gr.Slider( + label="ADetailer CFG scale" + suffix(n), + minimum=0.0, + maximum=30.0, + step=0.5, + value=7.0, + visible=True, + elem_id=eid("ad_cfg_scale"), + ) + + w.ad_use_cfg_scale.change( + gr_interactive, + inputs=w.ad_use_cfg_scale, + outputs=w.ad_cfg_scale, + queue=False, + ) + + with gr.Row(): + with gr.Column(variant="compact"): + w.ad_use_noise_multiplier = gr.Checkbox( + label="Use separate noise multiplier" + suffix(n), + value=False, + visible=True, + elem_id=eid("ad_use_noise_multiplier"), + ) + + w.ad_noise_multiplier = gr.Slider( + label="Noise multiplier for img2img" + suffix(n), + minimum=0.5, + maximum=1.5, + step=0.01, + value=1.0, + visible=True, + elem_id=eid("ad_noise_multiplier"), + ) + + w.ad_use_noise_multiplier.change( + gr_interactive, + inputs=w.ad_use_noise_multiplier, + outputs=w.ad_noise_multiplier, + queue=False, + ) + + w.ad_restore_face = gr.Checkbox( + label="Restore faces after ADetailer" + suffix(n), + value=False, + elem_id=eid("ad_restore_face"), + ) + + +def controlnet(w: Widgets, n: int, is_img2img: bool): + eid = partial(elem_id, n=n, is_img2img=is_img2img) + cn_models = ["None", *get_cn_models()] + + with gr.Row(variant="panel"): + with gr.Column(variant="compact"): + w.ad_controlnet_model = gr.Dropdown( + label="ControlNet model" + suffix(n), + choices=cn_models, + value="None", + visible=True, + type="value", + interactive=controlnet_exists, + elem_id=eid("ad_controlnet_model"), + ) + + w.ad_controlnet_module = gr.Dropdown( + label="ControlNet module" + suffix(n), + choices=cn_module_choices, + value="inpaint_global_harmonious", + visible=False, + type="value", + interactive=controlnet_exists, + elem_id=eid("ad_controlnet_module"), + ) + + w.ad_controlnet_weight = gr.Slider( + label="ControlNet weight" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.01, + value=1.0, + visible=True, + interactive=controlnet_exists, + elem_id=eid("ad_controlnet_weight"), + ) + + w.ad_controlnet_model.change( + on_cn_model_update, + inputs=w.ad_controlnet_model, + outputs=w.ad_controlnet_module, + queue=False, + ) + + with gr.Column(variant="compact"): + w.ad_controlnet_guidance_start = gr.Slider( + label="ControlNet guidance start" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.01, + value=0.0, + visible=True, + interactive=controlnet_exists, + elem_id=eid("ad_controlnet_guidance_start"), + ) + + w.ad_controlnet_guidance_end = gr.Slider( + label="ControlNet guidance end" + suffix(n), + minimum=0.0, + maximum=1.0, + step=0.01, + value=1.0, + visible=True, + interactive=controlnet_exists, + elem_id=eid("ad_controlnet_guidance_end"), + ) diff --git a/extensions/adetailer/adetailer/ultralytics.py b/extensions/adetailer/adetailer/ultralytics.py new file mode 100644 index 0000000000000000000000000000000000000000..b44703efa4906ec34673a246d4cdbbd1c711a589 --- /dev/null +++ b/extensions/adetailer/adetailer/ultralytics.py @@ -0,0 +1,54 @@ +from __future__ import annotations + +from pathlib import Path + +import cv2 +from PIL import Image + +from adetailer import PredictOutput +from adetailer.common import create_mask_from_bbox + + +def ultralytics_predict( + model_path: str | Path, + image: Image.Image, + confidence: float = 0.3, + device: str = "", +) -> PredictOutput: + from ultralytics import YOLO + + model_path = str(model_path) + + model = YOLO(model_path) + pred = model(image, conf=confidence, device=device) + + bboxes = pred[0].boxes.xyxy.cpu().numpy() + if bboxes.size == 0: + return PredictOutput() + bboxes = bboxes.tolist() + + if pred[0].masks is None: + masks = create_mask_from_bbox(bboxes, image.size) + else: + masks = mask_to_pil(pred[0].masks.data, image.size) + preview = pred[0].plot() + preview = cv2.cvtColor(preview, cv2.COLOR_BGR2RGB) + preview = Image.fromarray(preview) + + return PredictOutput(bboxes=bboxes, masks=masks, preview=preview) + + +def mask_to_pil(masks, shape: tuple[int, int]) -> list[Image.Image]: + """ + Parameters + ---------- + masks: torch.Tensor, dtype=torch.float32, shape=(N, H, W). + The device can be CUDA, but `to_pil_image` takes care of that. + + shape: tuple[int, int] + (width, height) of the original image + """ + from torchvision.transforms.functional import to_pil_image + + n = masks.shape[0] + return [to_pil_image(masks[i], mode="L").resize(shape) for i in range(n)] diff --git a/extensions/adetailer/controlnet_ext/__init__.py b/extensions/adetailer/controlnet_ext/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..0ab666835157561426d684d798735e724a5a4dbe --- /dev/null +++ b/extensions/adetailer/controlnet_ext/__init__.py @@ -0,0 +1,7 @@ +from .controlnet_ext import ControlNetExt, controlnet_exists, get_cn_models + +__all__ = [ + "ControlNetExt", + "controlnet_exists", + "get_cn_models", +] diff --git a/extensions/adetailer/controlnet_ext/__pycache__/__init__.cpython-310.pyc b/extensions/adetailer/controlnet_ext/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..56261a698ae0559226b05ccd47810b008f09b771 Binary files /dev/null and b/extensions/adetailer/controlnet_ext/__pycache__/__init__.cpython-310.pyc differ diff --git a/extensions/adetailer/controlnet_ext/__pycache__/controlnet_ext.cpython-310.pyc b/extensions/adetailer/controlnet_ext/__pycache__/controlnet_ext.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d9baf18f892d4fa732ccbad749d404566b6dcd45 Binary files /dev/null and b/extensions/adetailer/controlnet_ext/__pycache__/controlnet_ext.cpython-310.pyc differ diff --git a/extensions/adetailer/controlnet_ext/__pycache__/restore.cpython-310.pyc b/extensions/adetailer/controlnet_ext/__pycache__/restore.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5403c89d58d355e68234eda6abc511f58a7a7249 Binary files /dev/null and b/extensions/adetailer/controlnet_ext/__pycache__/restore.cpython-310.pyc differ diff --git a/extensions/adetailer/controlnet_ext/controlnet_ext.py b/extensions/adetailer/controlnet_ext/controlnet_ext.py new file mode 100644 index 0000000000000000000000000000000000000000..f0c89187c7f670f38dfc70a7b30ef293854191af --- /dev/null +++ b/extensions/adetailer/controlnet_ext/controlnet_ext.py @@ -0,0 +1,140 @@ +from __future__ import annotations + +import importlib +import re +from functools import lru_cache +from pathlib import Path + +from modules import extensions, sd_models, shared +from modules.paths import data_path, models_path, script_path + +ext_path = Path(data_path, "extensions") +ext_builtin_path = Path(script_path, "extensions-builtin") +controlnet_exists = False +controlnet_path = None +cn_base_path = "" + +for extension in extensions.active(): + if not extension.enabled: + continue + # For cases like sd-webui-controlnet-master + if "sd-webui-controlnet" in extension.name: + controlnet_exists = True + controlnet_path = Path(extension.path) + cn_base_path = ".".join(controlnet_path.parts[-2:]) + break + +cn_model_module = { + "inpaint": "inpaint_global_harmonious", + "scribble": "t2ia_sketch_pidi", + "lineart": "lineart_coarse", + "openpose": "openpose_full", + "tile": None, +} +cn_model_regex = re.compile("|".join(cn_model_module.keys())) + + +class ControlNetExt: + def __init__(self): + self.cn_models = ["None"] + self.cn_available = False + self.external_cn = None + + def init_controlnet(self): + import_path = cn_base_path + ".scripts.external_code" + + self.external_cn = importlib.import_module(import_path, "external_code") + self.cn_available = True + models = self.external_cn.get_models() + self.cn_models.extend(m for m in models if cn_model_regex.search(m)) + + def update_scripts_args( + self, + p, + model: str, + module: str | None, + weight: float, + guidance_start: float, + guidance_end: float, + ): + if (not self.cn_available) or model == "None": + return + + if module is None: + for m, v in cn_model_module.items(): + if m in model: + module = v + break + + cn_units = [ + self.external_cn.ControlNetUnit( + model=model, + weight=weight, + control_mode=self.external_cn.ControlMode.BALANCED, + module=module, + guidance_start=guidance_start, + guidance_end=guidance_end, + pixel_perfect=True, + ) + ] + + self.external_cn.update_cn_script_in_processing(p, cn_units) + + +def get_cn_model_dirs() -> list[Path]: + cn_model_dir = Path(models_path, "ControlNet") + if controlnet_path is not None: + cn_model_dir_old = controlnet_path.joinpath("models") + else: + cn_model_dir_old = None + ext_dir1 = shared.opts.data.get("control_net_models_path", "") + ext_dir2 = shared.opts.data.get("controlnet_dir", "") + + dirs = [cn_model_dir] + for ext_dir in [cn_model_dir_old, ext_dir1, ext_dir2]: + if ext_dir: + dirs.append(Path(ext_dir)) + + return dirs + + +@lru_cache +def _get_cn_models() -> list[str]: + """ + Since we can't import ControlNet, we use a function that does something like + controlnet's `list(global_state.cn_models_names.values())`. + """ + cn_model_exts = (".pt", ".pth", ".ckpt", ".safetensors") + dirs = get_cn_model_dirs() + name_filter = shared.opts.data.get("control_net_models_name_filter", "") + name_filter = name_filter.strip(" ").lower() + + model_paths = [] + + for base in dirs: + if not base.exists(): + continue + + for p in base.rglob("*"): + if ( + p.is_file() + and p.suffix in cn_model_exts + and cn_model_regex.search(p.name) + ): + if name_filter and name_filter not in p.name.lower(): + continue + model_paths.append(p) + model_paths.sort(key=lambda p: p.name) + + models = [] + for p in model_paths: + model_hash = sd_models.model_hash(p) + name = f"{p.stem} [{model_hash}]" + models.append(name) + return models + + +def get_cn_models() -> list[str]: + if controlnet_exists: + return _get_cn_models() + return [] diff --git a/extensions/adetailer/controlnet_ext/restore.py b/extensions/adetailer/controlnet_ext/restore.py new file mode 100644 index 0000000000000000000000000000000000000000..5b9bfa6292c46c2f940c1723411494584627e9d8 --- /dev/null +++ b/extensions/adetailer/controlnet_ext/restore.py @@ -0,0 +1,49 @@ +from __future__ import annotations + +from contextlib import contextmanager + +from modules import img2img, processing, shared + + +def cn_restore_unet_hook(p, cn_latest_network): + if cn_latest_network is not None: + unet = p.sd_model.model.diffusion_model + cn_latest_network.restore(unet) + + +class CNHijackRestore: + def __init__(self): + self.process = hasattr(processing, "__controlnet_original_process_images_inner") + self.img2img = hasattr(img2img, "__controlnet_original_process_batch") + + def __enter__(self): + if self.process: + self.orig_process = processing.process_images_inner + processing.process_images_inner = getattr( + processing, "__controlnet_original_process_images_inner" + ) + if self.img2img: + self.orig_img2img = img2img.process_batch + img2img.process_batch = getattr( + img2img, "__controlnet_original_process_batch" + ) + + def __exit__(self, *args, **kwargs): + if self.process: + processing.process_images_inner = self.orig_process + if self.img2img: + img2img.process_batch = self.orig_img2img + + +@contextmanager +def cn_allow_script_control(): + orig = False + if "control_net_allow_script_control" in shared.opts.data: + try: + orig = shared.opts.data["control_net_allow_script_control"] + shared.opts.data["control_net_allow_script_control"] = True + yield + finally: + shared.opts.data["control_net_allow_script_control"] = orig + else: + yield diff --git a/extensions/adetailer/install.py b/extensions/adetailer/install.py new file mode 100644 index 0000000000000000000000000000000000000000..25bfba2b16762f96fc5caedc55de261d3c2523db --- /dev/null +++ b/extensions/adetailer/install.py @@ -0,0 +1,81 @@ +from __future__ import annotations + +import importlib.util +import subprocess +import sys +from importlib.metadata import version # python >= 3.8 + +from packaging.version import parse + + +def is_installed( + package: str, min_version: str | None = None, max_version: str | None = None +): + try: + spec = importlib.util.find_spec(package) + except ModuleNotFoundError: + return False + + if spec is None: + return False + + if not min_version and not max_version: + return True + + if not min_version: + min_version = "0.0.0" + if not max_version: + max_version = "99999999.99999999.99999999" + + if package == "google.protobuf": + package = "protobuf" + + try: + pkg_version = version(package) + return parse(min_version) <= parse(pkg_version) <= parse(max_version) + except Exception: + return False + + +def run_pip(*args): + subprocess.run([sys.executable, "-m", "pip", "install", *args]) + + +def install(): + deps = [ + # requirements + ("ultralytics", "8.0.97", None), + ("mediapipe", "0.10.0", None), + ("huggingface_hub", None, None), + ("pydantic", "1.10.8", None), + ("rich", "13.4.2", None), + # mediapipe + ("protobuf", "3.20.0", "3.20.9999"), + ] + + for pkg, low, high in deps: + # https://github.com/protocolbuffers/protobuf/tree/main/python + name = "google.protobuf" if pkg == "protobuf" else pkg + + if not is_installed(name, low, high): + if low and high: + cmd = f"{pkg}>={low},<={high}" + elif low: + cmd = f"{pkg}>={low}" + elif high: + cmd = f"{pkg}<={high}" + else: + cmd = pkg + + run_pip("-U", cmd) + + +try: + import launch + + skip_install = launch.args.skip_install +except Exception: + skip_install = False + +if not skip_install: + install() diff --git a/extensions/adetailer/preload.py b/extensions/adetailer/preload.py new file mode 100644 index 0000000000000000000000000000000000000000..10be161f22b0a5ef7083609829a21b547eae9aea --- /dev/null +++ b/extensions/adetailer/preload.py @@ -0,0 +1,9 @@ +import argparse + + +def preload(parser: argparse.ArgumentParser): + parser.add_argument( + "--ad-no-huggingface", + action="store_true", + help="Don't use adetailer models from huggingface", + ) diff --git a/extensions/adetailer/pyproject.toml b/extensions/adetailer/pyproject.toml new file mode 100644 index 0000000000000000000000000000000000000000..be1fc0c9bea946d9d1dabd2514f573b5117ae683 --- /dev/null +++ b/extensions/adetailer/pyproject.toml @@ -0,0 +1,26 @@ +[project] +name = "adetailer" +description = "An object detection and auto-mask extension for stable diffusion webui." +authors = [ + {name = "dowon", email = "ks2515@naver.com"}, +] +requires-python = ">=3.8,<3.12" +readme = "README.md" +license = {text = "AGPL-3.0"} + +[project.urls] +repository = "https://github.com/Bing-su/adetailer" + +[tool.isort] +profile = "black" +known_first_party = ["launch", "modules"] + +[tool.ruff] +select = ["A", "B", "C4", "C90", "E", "EM", "F", "FA", "I001", "ISC", "N", "PIE", "PT", "RET", "RUF", "SIM", "UP", "W"] +ignore = ["B008", "B905", "E501", "F401", "UP007"] + +[tool.ruff.isort] +known-first-party = ["launch", "modules"] + +[tool.ruff.per-file-ignores] +"sd_webui/*.py" = ["B027", "F403"] diff --git a/extensions/adetailer/scripts/!adetailer.py b/extensions/adetailer/scripts/!adetailer.py new file mode 100644 index 0000000000000000000000000000000000000000..ed5de1a690068494491a1c485208ac59bd5af0e4 --- /dev/null +++ b/extensions/adetailer/scripts/!adetailer.py @@ -0,0 +1,784 @@ +from __future__ import annotations + +import os +import platform +import re +import sys +import traceback +from contextlib import contextmanager +from copy import copy, deepcopy +from functools import partial +from pathlib import Path +from textwrap import dedent +from typing import Any + +import gradio as gr +import torch +from rich import print + +import modules +from adetailer import ( + AFTER_DETAILER, + __version__, + get_models, + mediapipe_predict, + ultralytics_predict, +) +from adetailer.args import ALL_ARGS, BBOX_SORTBY, ADetailerArgs, EnableChecker +from adetailer.common import PredictOutput +from adetailer.mask import filter_by_ratio, mask_preprocess, sort_bboxes +from adetailer.traceback import rich_traceback +from adetailer.ui import adui, ordinal, suffix +from controlnet_ext import ControlNetExt, controlnet_exists, get_cn_models +from controlnet_ext.restore import ( + CNHijackRestore, + cn_allow_script_control, + cn_restore_unet_hook, +) +from sd_webui import images, safe, script_callbacks, scripts, shared +from sd_webui.devices import NansException +from sd_webui.paths import data_path, models_path +from sd_webui.processing import ( + Processed, + StableDiffusionProcessingImg2Img, + create_infotext, + process_images, +) +from sd_webui.shared import cmd_opts, opts, state + +no_huggingface = getattr(cmd_opts, "ad_no_huggingface", False) +adetailer_dir = Path(models_path, "adetailer") +model_mapping = get_models(adetailer_dir, huggingface=not no_huggingface) +txt2img_submit_button = img2img_submit_button = None +SCRIPT_DEFAULT = "dynamic_prompting,dynamic_thresholding,wildcard_recursive,wildcards,lora_block_weight" + +if ( + not adetailer_dir.exists() + and adetailer_dir.parent.exists() + and os.access(adetailer_dir.parent, os.W_OK) +): + adetailer_dir.mkdir() + +print( + f"[-] ADetailer initialized. version: {__version__}, num models: {len(model_mapping)}" +) + + +@contextmanager +def change_torch_load(): + orig = torch.load + try: + torch.load = safe.unsafe_torch_load + yield + finally: + torch.load = orig + + +@contextmanager +def pause_total_tqdm(): + orig = opts.data.get("multiple_tqdm", True) + try: + opts.data["multiple_tqdm"] = False + yield + finally: + opts.data["multiple_tqdm"] = orig + + +@contextmanager +def preseve_prompts(p): + all_pt = copy(p.all_prompts) + all_ng = copy(p.all_negative_prompts) + try: + yield + finally: + p.all_prompts = all_pt + p.all_negative_prompts = all_ng + + +class AfterDetailerScript(scripts.Script): + def __init__(self): + super().__init__() + self.ultralytics_device = self.get_ultralytics_device() + + self.controlnet_ext = None + self.cn_script = None + self.cn_latest_network = None + + def __repr__(self): + return f"{self.__class__.__name__}(version={__version__})" + + def title(self): + return AFTER_DETAILER + + def show(self, is_img2img): + return scripts.AlwaysVisible + + def ui(self, is_img2img): + num_models = opts.data.get("ad_max_models", 2) + model_list = list(model_mapping.keys()) + + components, infotext_fields = adui( + num_models, + is_img2img, + model_list, + txt2img_submit_button, + img2img_submit_button, + ) + + self.infotext_fields = infotext_fields + return components + + def init_controlnet_ext(self) -> None: + if self.controlnet_ext is not None: + return + self.controlnet_ext = ControlNetExt() + + if controlnet_exists: + try: + self.controlnet_ext.init_controlnet() + except ImportError: + error = traceback.format_exc() + print( + f"[-] ADetailer: ControlNetExt init failed:\n{error}", + file=sys.stderr, + ) + + def update_controlnet_args(self, p, args: ADetailerArgs) -> None: + if self.controlnet_ext is None: + self.init_controlnet_ext() + + if ( + self.controlnet_ext is not None + and self.controlnet_ext.cn_available + and args.ad_controlnet_model != "None" + ): + self.controlnet_ext.update_scripts_args( + p, + model=args.ad_controlnet_model, + module=args.ad_controlnet_module, + weight=args.ad_controlnet_weight, + guidance_start=args.ad_controlnet_guidance_start, + guidance_end=args.ad_controlnet_guidance_end, + ) + + def is_ad_enabled(self, *args_) -> bool: + arg_list = [arg for arg in args_ if isinstance(arg, dict)] + if not args_ or not arg_list or not isinstance(args_[0], (bool, dict)): + message = f""" + [-] ADetailer: Invalid arguments passed to ADetailer. + input: {args_!r} + """ + raise ValueError(dedent(message)) + enable = args_[0] if isinstance(args_[0], bool) else True + checker = EnableChecker(enable=enable, arg_list=arg_list) + return checker.is_enabled() + + def get_args(self, p, *args_) -> list[ADetailerArgs]: + """ + `args_` is at least 1 in length by `is_ad_enabled` immediately above + """ + args = [arg for arg in args_ if isinstance(arg, dict)] + + if not args: + message = f"[-] ADetailer: Invalid arguments passed to ADetailer: {args_!r}" + raise ValueError(message) + + if hasattr(p, "adetailer_xyz"): + args[0].update(p.adetailer_xyz) + + all_inputs = [] + + for n, arg_dict in enumerate(args, 1): + try: + inp = ADetailerArgs(**arg_dict) + except ValueError as e: + msgs = [ + f"[-] ADetailer: ValidationError when validating {ordinal(n)} arguments: {e}\n" + ] + for attr in ALL_ARGS.attrs: + arg = arg_dict.get(attr) + dtype = type(arg) + arg = "DEFAULT" if arg is None else repr(arg) + msgs.append(f" {attr}: {arg} ({dtype})") + raise ValueError("\n".join(msgs)) from e + + all_inputs.append(inp) + + return all_inputs + + def extra_params(self, arg_list: list[ADetailerArgs]) -> dict: + params = {} + for n, args in enumerate(arg_list): + params.update(args.extra_params(suffix=suffix(n))) + params["ADetailer version"] = __version__ + return params + + @staticmethod + def get_ultralytics_device() -> str: + if "adetailer" in shared.cmd_opts.use_cpu: + return "cpu" + + if platform.system() == "Darwin": + return "" + + if any(getattr(cmd_opts, vram, False) for vram in ["lowvram", "medvram"]): + return "cpu" + + return "" + + def prompt_blank_replacement( + self, all_prompts: list[str], i: int, default: str + ) -> str: + if not all_prompts: + return default + if i < len(all_prompts): + return all_prompts[i] + j = i % len(all_prompts) + return all_prompts[j] + + def _get_prompt( + self, ad_prompt: str, all_prompts: list[str], i: int, default: str + ) -> list[str]: + prompts = re.split(r"\s*\[SEP\]\s*", ad_prompt) + blank_replacement = self.prompt_blank_replacement(all_prompts, i, default) + for n in range(len(prompts)): + if not prompts[n]: + prompts[n] = blank_replacement + return prompts + + def get_prompt(self, p, args: ADetailerArgs) -> tuple[list[str], list[str]]: + i = p._ad_idx + + prompt = self._get_prompt(args.ad_prompt, p.all_prompts, i, p.prompt) + negative_prompt = self._get_prompt( + args.ad_negative_prompt, p.all_negative_prompts, i, p.negative_prompt + ) + + return prompt, negative_prompt + + def get_seed(self, p) -> tuple[int, int]: + i = p._ad_idx + + if not p.all_seeds: + seed = p.seed + elif i < len(p.all_seeds): + seed = p.all_seeds[i] + else: + j = i % len(p.all_seeds) + seed = p.all_seeds[j] + + if not p.all_subseeds: + subseed = p.subseed + elif i < len(p.all_subseeds): + subseed = p.all_subseeds[i] + else: + j = i % len(p.all_subseeds) + subseed = p.all_subseeds[j] + + return seed, subseed + + def get_width_height(self, p, args: ADetailerArgs) -> tuple[int, int]: + if args.ad_use_inpaint_width_height: + width = args.ad_inpaint_width + height = args.ad_inpaint_height + else: + width = p.width + height = p.height + + return width, height + + def get_steps(self, p, args: ADetailerArgs) -> int: + if args.ad_use_steps: + return args.ad_steps + return p.steps + + def get_cfg_scale(self, p, args: ADetailerArgs) -> float: + if args.ad_use_cfg_scale: + return args.ad_cfg_scale + return p.cfg_scale + + def get_initial_noise_multiplier(self, p, args: ADetailerArgs) -> float | None: + if args.ad_use_noise_multiplier: + return args.ad_noise_multiplier + return None + + def infotext(self, p) -> str: + return create_infotext( + p, p.all_prompts, p.all_seeds, p.all_subseeds, None, 0, 0 + ) + + def write_params_txt(self, p) -> None: + infotext = self.infotext(p) + params_txt = Path(data_path, "params.txt") + params_txt.write_text(infotext, encoding="utf-8") + + def script_filter(self, p, args: ADetailerArgs): + script_runner = copy(p.scripts) + script_args = deepcopy(p.script_args) + self.disable_controlnet_units(script_args) + + ad_only_seleted_scripts = opts.data.get("ad_only_seleted_scripts", True) + if not ad_only_seleted_scripts: + return script_runner, script_args + + ad_script_names = opts.data.get("ad_script_names", SCRIPT_DEFAULT) + script_names_set = { + name + for script_name in ad_script_names.split(",") + for name in (script_name, script_name.strip()) + } + + if args.ad_controlnet_model != "None": + script_names_set.add("controlnet") + + filtered_alwayson = [] + for script_object in script_runner.alwayson_scripts: + filepath = script_object.filename + filename = Path(filepath).stem + if filename in script_names_set: + filtered_alwayson.append(script_object) + if filename == "controlnet": + self.cn_script = script_object + self.cn_latest_network = script_object.latest_network + + script_runner.alwayson_scripts = filtered_alwayson + return script_runner, script_args + + def disable_controlnet_units(self, script_args: list[Any]) -> None: + for obj in script_args: + if "controlnet" in obj.__class__.__name__.lower(): + if hasattr(obj, "enabled"): + obj.enabled = False + if hasattr(obj, "input_mode"): + obj.input_mode = getattr(obj.input_mode, "SIMPLE", "simple") + + elif isinstance(obj, dict) and "module" in obj: + obj["enabled"] = False + + def get_i2i_p(self, p, args: ADetailerArgs, image): + seed, subseed = self.get_seed(p) + width, height = self.get_width_height(p, args) + steps = self.get_steps(p, args) + cfg_scale = self.get_cfg_scale(p, args) + initial_noise_multiplier = self.get_initial_noise_multiplier(p, args) + + sampler_name = p.sampler_name + if sampler_name in ["PLMS", "UniPC"]: + sampler_name = "Euler" + + i2i = StableDiffusionProcessingImg2Img( + init_images=[image], + resize_mode=0, + denoising_strength=args.ad_denoising_strength, + mask=None, + mask_blur=args.ad_mask_blur, + inpainting_fill=1, + inpaint_full_res=args.ad_inpaint_only_masked, + inpaint_full_res_padding=args.ad_inpaint_only_masked_padding, + inpainting_mask_invert=0, + initial_noise_multiplier=initial_noise_multiplier, + sd_model=p.sd_model, + outpath_samples=p.outpath_samples, + outpath_grids=p.outpath_grids, + prompt="", # replace later + negative_prompt="", + styles=p.styles, + seed=seed, + subseed=subseed, + subseed_strength=p.subseed_strength, + seed_resize_from_h=p.seed_resize_from_h, + seed_resize_from_w=p.seed_resize_from_w, + sampler_name=sampler_name, + batch_size=1, + n_iter=1, + steps=steps, + cfg_scale=cfg_scale, + width=width, + height=height, + restore_faces=args.ad_restore_face, + tiling=p.tiling, + extra_generation_params=p.extra_generation_params, + do_not_save_samples=True, + do_not_save_grid=True, + ) + + i2i.cached_c = [None, None] + i2i.cached_uc = [None, None] + i2i.scripts, i2i.script_args = self.script_filter(p, args) + i2i._disable_adetailer = True + + if args.ad_controlnet_model != "None": + self.update_controlnet_args(i2i, args) + else: + i2i.control_net_enabled = False + + return i2i + + def save_image(self, p, image, *, condition: str, suffix: str) -> None: + i = p._ad_idx + if p.all_prompts: + i %= len(p.all_prompts) + save_prompt = p.all_prompts[i] + else: + save_prompt = p.prompt + seed, _ = self.get_seed(p) + + if opts.data.get(condition, False): + images.save_image( + image=image, + path=p.outpath_samples, + basename="", + seed=seed, + prompt=save_prompt, + extension=opts.samples_format, + info=self.infotext(p), + p=p, + suffix=suffix, + ) + + def get_ad_model(self, name: str): + if name not in model_mapping: + msg = f"[-] ADetailer: Model {name!r} not found. Available models: {list(model_mapping.keys())}" + raise ValueError(msg) + return model_mapping[name] + + def sort_bboxes(self, pred: PredictOutput) -> PredictOutput: + sortby = opts.data.get("ad_bbox_sortby", BBOX_SORTBY[0]) + sortby_idx = BBOX_SORTBY.index(sortby) + return sort_bboxes(pred, sortby_idx) + + def pred_preprocessing(self, pred: PredictOutput, args: ADetailerArgs): + pred = filter_by_ratio( + pred, low=args.ad_mask_min_ratio, high=args.ad_mask_max_ratio + ) + pred = self.sort_bboxes(pred) + return mask_preprocess( + pred.masks, + kernel=args.ad_dilate_erode, + x_offset=args.ad_x_offset, + y_offset=args.ad_y_offset, + merge_invert=args.ad_mask_merge_invert, + ) + + def i2i_prompts_replace( + self, i2i, prompts: list[str], negative_prompts: list[str], j: int + ) -> None: + i1 = min(j, len(prompts) - 1) + i2 = min(j, len(negative_prompts) - 1) + prompt = prompts[i1] + negative_prompt = negative_prompts[i2] + i2i.prompt = prompt + i2i.negative_prompt = negative_prompt + + @staticmethod + def compare_prompt(p, processed, n: int = 0): + if p.prompt != processed.all_prompts[0]: + print( + f"[-] ADetailer: applied {ordinal(n + 1)} ad_prompt: {processed.all_prompts[0]!r}" + ) + + if p.negative_prompt != processed.all_negative_prompts[0]: + print( + f"[-] ADetailer: applied {ordinal(n + 1)} ad_negative_prompt: {processed.all_negative_prompts[0]!r}" + ) + + def need_call_process(self, p) -> bool: + i = p._ad_idx + bs = p.batch_size + return i % bs == bs - 1 + + def need_call_postprocess(self, p) -> bool: + i = p._ad_idx + bs = p.batch_size + return i % bs == 0 + + @rich_traceback + def process(self, p, *args_): + if getattr(p, "_disable_adetailer", False): + return + + if self.is_ad_enabled(*args_): + arg_list = self.get_args(p, *args_) + extra_params = self.extra_params(arg_list) + p.extra_generation_params.update(extra_params) + + def _postprocess_image(self, p, pp, args: ADetailerArgs, *, n: int = 0) -> bool: + """ + Returns + ------- + bool + + `True` if image was processed, `False` otherwise. + """ + if state.interrupted: + return False + + i = p._ad_idx + + i2i = self.get_i2i_p(p, args, pp.image) + seed, subseed = self.get_seed(p) + ad_prompts, ad_negatives = self.get_prompt(p, args) + + is_mediapipe = args.ad_model.lower().startswith("mediapipe") + + kwargs = {} + if is_mediapipe: + predictor = mediapipe_predict + ad_model = args.ad_model + else: + predictor = ultralytics_predict + ad_model = self.get_ad_model(args.ad_model) + kwargs["device"] = self.ultralytics_device + + with change_torch_load(): + pred = predictor(ad_model, pp.image, args.ad_confidence, **kwargs) + + masks = self.pred_preprocessing(pred, args) + + if not masks: + print( + f"[-] ADetailer: nothing detected on image {i + 1} with {ordinal(n + 1)} settings." + ) + return False + + self.save_image( + p, + pred.preview, + condition="ad_save_previews", + suffix="-ad-preview" + suffix(n, "-"), + ) + + steps = len(masks) + processed = None + state.job_count += steps + + if is_mediapipe: + print(f"mediapipe: {steps} detected.") + + _user_pt = p.prompt + _user_ng = p.negative_prompt + + p2 = copy(i2i) + for j in range(steps): + p2.image_mask = masks[j] + self.i2i_prompts_replace(p2, ad_prompts, ad_negatives, j) + + if re.match(r"^\s*\[SKIP\]\s*$", p2.prompt): + continue + + p2.seed = seed + j + p2.subseed = subseed + j + + try: + processed = process_images(p2) + except NansException as e: + msg = f"[-] ADetailer: 'NansException' occurred with {ordinal(n + 1)} settings.\n{e}" + print(msg, file=sys.stderr) + continue + finally: + p2.close() + + self.compare_prompt(p2, processed, n=n) + p2 = copy(i2i) + p2.init_images = [processed.images[0]] + + if processed is not None: + pp.image = processed.images[0] + return True + + return False + + @rich_traceback + def postprocess_image(self, p, pp, *args_): + if getattr(p, "_disable_adetailer", False): + return + + if not self.is_ad_enabled(*args_): + return + + p._ad_idx = getattr(p, "_ad_idx", -1) + 1 + init_image = copy(pp.image) + arg_list = self.get_args(p, *args_) + + if p.scripts is not None and self.need_call_postprocess(p): + dummy = Processed(p, [], p.seed, "") + with preseve_prompts(p): + p.scripts.postprocess(copy(p), dummy) + + is_processed = False + with CNHijackRestore(), pause_total_tqdm(), cn_allow_script_control(): + for n, args in enumerate(arg_list): + if args.ad_model == "None": + continue + is_processed |= self._postprocess_image(p, pp, args, n=n) + + if is_processed: + self.save_image( + p, init_image, condition="ad_save_images_before", suffix="-ad-before" + ) + + if p.scripts is not None and self.need_call_process(p): + with preseve_prompts(p): + p.scripts.process(copy(p)) + + try: + ia = p._ad_idx + lenp = len(p.all_prompts) + if ia % lenp == lenp - 1: + self.write_params_txt(p) + except Exception: + pass + + +def on_after_component(component, **_kwargs): + global txt2img_submit_button, img2img_submit_button + if getattr(component, "elem_id", None) == "txt2img_generate": + txt2img_submit_button = component + return + + if getattr(component, "elem_id", None) == "img2img_generate": + img2img_submit_button = component + + +def on_ui_settings(): + section = ("ADetailer", AFTER_DETAILER) + shared.opts.add_option( + "ad_max_models", + shared.OptionInfo( + default=2, + label="Max models", + component=gr.Slider, + component_args={"minimum": 1, "maximum": 10, "step": 1}, + section=section, + ), + ) + + shared.opts.add_option( + "ad_save_previews", + shared.OptionInfo(False, "Save mask previews", section=section), + ) + + shared.opts.add_option( + "ad_save_images_before", + shared.OptionInfo(False, "Save images before ADetailer", section=section), + ) + + shared.opts.add_option( + "ad_only_seleted_scripts", + shared.OptionInfo( + True, "Apply only selected scripts to ADetailer", section=section + ), + ) + + textbox_args = { + "placeholder": "comma-separated list of script names", + "interactive": True, + } + + shared.opts.add_option( + "ad_script_names", + shared.OptionInfo( + default=SCRIPT_DEFAULT, + label="Script names to apply to ADetailer (separated by comma)", + component=gr.Textbox, + component_args=textbox_args, + section=section, + ), + ) + + shared.opts.add_option( + "ad_bbox_sortby", + shared.OptionInfo( + default="None", + label="Sort bounding boxes by", + component=gr.Radio, + component_args={"choices": BBOX_SORTBY}, + section=section, + ), + ) + + +# xyz_grid + + +def make_axis_on_xyz_grid(): + xyz_grid = None + for script in scripts.scripts_data: + if script.script_class.__module__ == "xyz_grid.py": + xyz_grid = script.module + break + + if xyz_grid is None: + return + + model_list = ["None", *model_mapping.keys()] + + def set_value(p, x, xs, *, field: str): + if not hasattr(p, "adetailer_xyz"): + p.adetailer_xyz = {} + p.adetailer_xyz[field] = x + + axis = [ + xyz_grid.AxisOption( + "[ADetailer] ADetailer model 1st", + str, + partial(set_value, field="ad_model"), + choices=lambda: model_list, + ), + xyz_grid.AxisOption( + "[ADetailer] ADetailer prompt 1st", + str, + partial(set_value, field="ad_prompt"), + ), + xyz_grid.AxisOption( + "[ADetailer] ADetailer negative prompt 1st", + str, + partial(set_value, field="ad_negative_prompt"), + ), + xyz_grid.AxisOption( + "[ADetailer] Mask erosion / dilation 1st", + int, + partial(set_value, field="ad_dilate_erode"), + ), + xyz_grid.AxisOption( + "[ADetailer] Inpaint denoising strength 1st", + float, + partial(set_value, field="ad_denoising_strength"), + ), + xyz_grid.AxisOption( + "[ADetailer] Inpaint only masked 1st", + str, + partial(set_value, field="ad_inpaint_only_masked"), + choices=lambda: ["True", "False"], + ), + xyz_grid.AxisOption( + "[ADetailer] Inpaint only masked padding 1st", + int, + partial(set_value, field="ad_inpaint_only_masked_padding"), + ), + xyz_grid.AxisOption( + "[ADetailer] ControlNet model 1st", + str, + partial(set_value, field="ad_controlnet_model"), + choices=lambda: ["None", *get_cn_models()], + ), + ] + + if not any(x.label.startswith("[ADetailer]") for x in xyz_grid.axis_options): + xyz_grid.axis_options.extend(axis) + + +def on_before_ui(): + try: + make_axis_on_xyz_grid() + except Exception: + error = traceback.format_exc() + print( + f"[-] ADetailer: xyz_grid error:\n{error}", + file=sys.stderr, + ) + + +script_callbacks.on_ui_settings(on_ui_settings) +script_callbacks.on_after_component(on_after_component) +script_callbacks.on_before_ui(on_before_ui) diff --git a/extensions/adetailer/scripts/__pycache__/!adetailer.cpython-310.pyc b/extensions/adetailer/scripts/__pycache__/!adetailer.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..0d1ca898a756d7abe89a284fc255c02135eec9be Binary files /dev/null and b/extensions/adetailer/scripts/__pycache__/!adetailer.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__init__.py b/extensions/adetailer/sd_webui/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/extensions/adetailer/sd_webui/__pycache__/__init__.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..2eb822780712120c19b581cb4932b2f2c54ffbd4 Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/__init__.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/devices.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/devices.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d5ede54f1b392c5e9ab99e0df7ec15725831fcdb Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/devices.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/images.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/images.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a3ab148994c4261a772d88c9d9b95b52c9d1b6fb Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/images.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/paths.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/paths.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b3a80c07c80e318fc3e685b9d279f4044d6a9b04 Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/paths.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/processing.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/processing.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..29ee4e968542e481b28a156c38100c3df3d26f2d Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/processing.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/safe.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/safe.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..10864f1ed088f79db41122f9652291bbc6babd4d Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/safe.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/script_callbacks.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/script_callbacks.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1f3a812f9aa925891410738f28cff2105c637e5e Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/script_callbacks.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/scripts.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/scripts.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e0ce9731bc08d042b181fff95aaed4e6312f07e6 Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/scripts.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/__pycache__/shared.cpython-310.pyc b/extensions/adetailer/sd_webui/__pycache__/shared.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..75004932ff0ae59e2d76be8a2e8d71ab7de987f9 Binary files /dev/null and b/extensions/adetailer/sd_webui/__pycache__/shared.cpython-310.pyc differ diff --git a/extensions/adetailer/sd_webui/devices.py b/extensions/adetailer/sd_webui/devices.py new file mode 100644 index 0000000000000000000000000000000000000000..51d0569a8ea9dda869ac43ac338835e2fbb27782 --- /dev/null +++ b/extensions/adetailer/sd_webui/devices.py @@ -0,0 +1,11 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + + class NansException(Exception): # noqa: N818 + pass + +else: + from modules.devices import NansException diff --git a/extensions/adetailer/sd_webui/images.py b/extensions/adetailer/sd_webui/images.py new file mode 100644 index 0000000000000000000000000000000000000000..b4a2dbce09ad3583f3aac4624c969657103c145e --- /dev/null +++ b/extensions/adetailer/sd_webui/images.py @@ -0,0 +1,62 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from PIL import Image, PngImagePlugin + + from sd_webui.processing import StableDiffusionProcessing + + def save_image( + image: Image.Image, + path: str, + basename: str, + seed: int | None = None, + prompt: str = "", + extension: str = "png", + info: str | PngImagePlugin.iTXt = "", + short_filename: bool = False, + no_prompt: bool = False, + grid: bool = False, + pnginfo_section_name: str = "parameters", + p: StableDiffusionProcessing | None = None, + existing_info: dict | None = None, + forced_filename: str | None = None, + suffix: str = "", + save_to_dirs: bool = False, + ) -> tuple[str, str | None]: + """Save an image. + + Args: + image (`PIL.Image`): + The image to be saved. + path (`str`): + The directory to save the image. Note, the option `save_to_dirs` will make the image to be saved into a sub directory. + basename (`str`): + The base filename which will be applied to `filename pattern`. + seed, prompt, short_filename, + extension (`str`): + Image file extension, default is `png`. + pngsectionname (`str`): + Specify the name of the section which `info` will be saved in. + info (`str` or `PngImagePlugin.iTXt`): + PNG info chunks. + existing_info (`dict`): + Additional PNG info. `existing_info == {pngsectionname: info, ...}` + no_prompt: + TODO I don't know its meaning. + p (`StableDiffusionProcessing`) + forced_filename (`str`): + If specified, `basename` and filename pattern will be ignored. + save_to_dirs (bool): + If true, the image will be saved into a subdirectory of `path`. + + Returns: (fullfn, txt_fullfn) + fullfn (`str`): + The full path of the saved imaged. + txt_fullfn (`str` or None): + If a text file is saved for this image, this will be its full path. Otherwise None. + """ + +else: + from modules.images import save_image diff --git a/extensions/adetailer/sd_webui/paths.py b/extensions/adetailer/sd_webui/paths.py new file mode 100644 index 0000000000000000000000000000000000000000..8050ba080788f291a3c717016798f3c6531e655e --- /dev/null +++ b/extensions/adetailer/sd_webui/paths.py @@ -0,0 +1,14 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import os + + models_path = os.path.join(os.path.dirname(__file__), "1") + script_path = os.path.join(os.path.dirname(__file__), "2") + data_path = os.path.join(os.path.dirname(__file__), "3") + extensions_dir = os.path.join(os.path.dirname(__file__), "4") + extensions_builtin_dir = os.path.join(os.path.dirname(__file__), "5") +else: + from modules.paths import data_path, models_path, script_path diff --git a/extensions/adetailer/sd_webui/processing.py b/extensions/adetailer/sd_webui/processing.py new file mode 100644 index 0000000000000000000000000000000000000000..e65d99f9ab44cb748075455d6ea1c02a481b6f0b --- /dev/null +++ b/extensions/adetailer/sd_webui/processing.py @@ -0,0 +1,176 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from dataclasses import dataclass, field + from typing import Any, Callable + + import numpy as np + import torch + from PIL import Image + + def _image(): + return Image.new("L", (512, 512)) + + @dataclass + class StableDiffusionProcessing: + sd_model: torch.nn.Module = field(default_factory=lambda: torch.nn.Linear(1, 1)) + outpath_samples: str = "" + outpath_grids: str = "" + prompt: str = "" + prompt_for_display: str = "" + negative_prompt: str = "" + styles: list[str] = field(default_factory=list) + seed: int = -1 + subseed: int = -1 + subseed_strength: float = 0.0 + seed_resize_from_h: int = -1 + seed_resize_from_w: int = -1 + sampler_name: str | None = None + batch_size: int = 1 + n_iter: int = 1 + steps: int = 50 + cfg_scale: float = 7.0 + width: int = 512 + height: int = 512 + restore_faces: bool = False + tiling: bool = False + do_not_save_samples: bool = False + do_not_save_grid: bool = False + extra_generation_params: dict[str, Any] = field(default_factory=dict) + overlay_images: list[Image.Image] = field(default_factory=list) + eta: float = 0.0 + do_not_reload_embeddings: bool = False + paste_to: tuple[int | float, ...] = (0, 0, 0, 0) + color_corrections: list[np.ndarray] = field(default_factory=list) + denoising_strength: float = 0.0 + sampler_noise_scheduler_override: Callable | None = None + ddim_discretize: str = "" + s_min_uncond: float = 0.0 + s_churn: float = 0.0 + s_tmin: float = 0.0 + s_tmax: float = 0.0 + s_noise: float = 0.0 + override_settings: dict[str, Any] = field(default_factory=dict) + override_settings_restore_afterwards: bool = False + is_using_inpainting_conditioning: bool = False + disable_extra_networks: bool = False + scripts: Any = None + script_args: list[Any] = field(default_factory=list) + all_prompts: list[str] = field(default_factory=list) + all_negative_prompts: list[str] = field(default_factory=list) + all_seeds: list[int] = field(default_factory=list) + all_subseeds: list[int] = field(default_factory=list) + iteration: int = 1 + is_hr_pass: bool = False + + def close(self) -> None: + pass + + @dataclass + class StableDiffusionProcessingTxt2Img(StableDiffusionProcessing): + sampler: Callable | None = None + enable_hr: bool = False + denoising_strength: float = 0.75 + hr_scale: float = 2.0 + hr_upscaler: str = "" + hr_second_pass_steps: int = 0 + hr_resize_x: int = 0 + hr_resize_y: int = 0 + hr_upscale_to_x: int = 0 + hr_upscale_to_y: int = 0 + width: int = 512 + height: int = 512 + truncate_x: int = 512 + truncate_y: int = 512 + applied_old_hires_behavior_to: tuple[int, int] = (512, 512) + + @dataclass + class StableDiffusionProcessingImg2Img(StableDiffusionProcessing): + sampler: Callable | None = None + init_images: list[Image.Image] = field(default_factory=list) + resize_mode: int = 0 + denoising_strength: float = 0.75 + image_cfg_scale: float | None = None + init_latent: torch.Tensor | None = None + image_mask: Image.Image = field(default_factory=_image) + latent_mask: Image.Image = field(default_factory=_image) + mask_for_overlay: Image.Image = field(default_factory=_image) + mask_blur: int = 4 + inpainting_fill: int = 0 + inpaint_full_res: bool = True + inpaint_full_res_padding: int = 0 + inpainting_mask_invert: int | bool = 0 + initial_noise_multiplier: float = 1.0 + mask: torch.Tensor | None = None + nmask: torch.Tensor | None = None + image_conditioning: torch.Tensor | None = None + + @dataclass + class Processed: + images: list[Image.Image] = field(default_factory=list) + prompt: list[str] = field(default_factory=list) + negative_prompt: list[str] = field(default_factory=list) + seed: list[int] = field(default_factory=list) + subseed: list[int] = field(default_factory=list) + subseed_strength: float = 0.0 + info: str = "" + comments: str = "" + width: int = 512 + height: int = 512 + sampler_name: str = "" + cfg_scale: float = 7.0 + image_cfg_scale: float | None = None + steps: int = 50 + batch_size: int = 1 + restore_faces: bool = False + face_restoration_model: str | None = None + sd_model_hash: str = "" + seed_resize_from_w: int = -1 + seed_resize_from_h: int = -1 + denoising_strength: float = 0.0 + extra_generation_params: dict[str, Any] = field(default_factory=dict) + index_of_first_image: int = 0 + styles: list[str] = field(default_factory=list) + job_timestamp: str = "" + clip_skip: int = 1 + eta: float = 0.0 + ddim_discretize: str = "" + s_churn: float = 0.0 + s_tmin: float = 0.0 + s_tmax: float = 0.0 + s_noise: float = 0.0 + sampler_noise_scheduler_override: Callable | None = None + is_using_inpainting_conditioning: bool = False + all_prompts: list[str] = field(default_factory=list) + all_negative_prompts: list[str] = field(default_factory=list) + all_seeds: list[int] = field(default_factory=list) + all_subseeds: list[int] = field(default_factory=list) + infotexts: list[str] = field(default_factory=list) + + def create_infotext( + p: StableDiffusionProcessingTxt2Img | StableDiffusionProcessingImg2Img, + all_prompts: list[str], + all_seeds: list[int], + all_subseeds: list[int], + comments: Any, + iteration: int = 0, + position_in_batch: int = 0, + ) -> str: + pass + + def process_images( + p: StableDiffusionProcessingTxt2Img | StableDiffusionProcessingImg2Img, + ) -> Processed: + pass + +else: + from modules.processing import ( + Processed, + StableDiffusionProcessing, + StableDiffusionProcessingImg2Img, + StableDiffusionProcessingTxt2Img, + create_infotext, + process_images, + ) diff --git a/extensions/adetailer/sd_webui/safe.py b/extensions/adetailer/sd_webui/safe.py new file mode 100644 index 0000000000000000000000000000000000000000..c17a1b97d3ce1590d0ecfa49373a826a300404d1 --- /dev/null +++ b/extensions/adetailer/sd_webui/safe.py @@ -0,0 +1,10 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import torch + + unsafe_torch_load = torch.load +else: + from modules.safe import unsafe_torch_load diff --git a/extensions/adetailer/sd_webui/script_callbacks.py b/extensions/adetailer/sd_webui/script_callbacks.py new file mode 100644 index 0000000000000000000000000000000000000000..ebb3ac0507bd32a8b6f962e9796829f539cd31a1 --- /dev/null +++ b/extensions/adetailer/sd_webui/script_callbacks.py @@ -0,0 +1,26 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from typing import Callable + + def on_app_started(callback: Callable): + pass + + def on_ui_settings(callback: Callable): + pass + + def on_after_component(callback: Callable): + pass + + def on_before_ui(callback: Callable): + pass + +else: + from modules.script_callbacks import ( + on_after_component, + on_app_started, + on_before_ui, + on_ui_settings, + ) diff --git a/extensions/adetailer/sd_webui/scripts.py b/extensions/adetailer/sd_webui/scripts.py new file mode 100644 index 0000000000000000000000000000000000000000..e515bbb70e5e6f880892b80a9af62f96efe87dea --- /dev/null +++ b/extensions/adetailer/sd_webui/scripts.py @@ -0,0 +1,94 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from abc import ABC, abstractmethod + from collections import namedtuple + from dataclasses import dataclass + from typing import Any + + import gradio as gr + from PIL import Image + + from sd_webui.processing import ( + Processed, + StableDiffusionProcessingImg2Img, + StableDiffusionProcessingTxt2Img, + ) + + SDPType = StableDiffusionProcessingImg2Img | StableDiffusionProcessingTxt2Img + AlwaysVisible = object() + + @dataclass + class PostprocessImageArgs: + image: Image.Image + + class Script(ABC): + filename: str + args_from: int + args_to: int + alwayson: bool + + is_txt2img: bool + is_img2img: bool + + group: gr.Group + infotext_fields: list[tuple[str, str]] + paste_field_names: list[str] + + @abstractmethod + def title(self): + raise NotImplementedError + + def ui(self, is_img2img: bool): + pass + + def show(self, is_img2img: bool): + return True + + def run(self, p: SDPType, *args): + pass + + def process(self, p: SDPType, *args): + pass + + def before_process_batch(self, p: SDPType, *args, **kwargs): + pass + + def process_batch(self, p: SDPType, *args, **kwargs): + pass + + def postprocess_batch(self, p: SDPType, *args, **kwargs): + pass + + def postprocess_image(self, p: SDPType, pp: PostprocessImageArgs, *args): + pass + + def postprocess(self, p: SDPType, processed: Processed, *args): + pass + + def before_component(self, component, **kwargs): + pass + + def after_component(self, component, **kwargs): + pass + + def describe(self): + return "" + + def elem_id(self, item_id: Any) -> str: + pass + + ScriptClassData = namedtuple( + "ScriptClassData", ["script_class", "path", "basedir", "module"] + ) + scripts_data: list[ScriptClassData] = [] + +else: + from modules.scripts import ( + AlwaysVisible, + PostprocessImageArgs, + Script, + scripts_data, + ) diff --git a/extensions/adetailer/sd_webui/shared.py b/extensions/adetailer/sd_webui/shared.py new file mode 100644 index 0000000000000000000000000000000000000000..18b0cd0801e652340c86dc6e7074fb0c92b99cc4 --- /dev/null +++ b/extensions/adetailer/sd_webui/shared.py @@ -0,0 +1,66 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import argparse + from dataclasses import dataclass + from typing import Any, Callable + + import torch + from PIL import Image + + @dataclass + class State: + skipped: bool = False + interrupted: bool = False + job: str = "" + job_no: int = 0 + job_count: int = 0 + processing_has_refined_job_count: bool = False + job_timestamp: str = "0" + sampling_step: int = 0 + sampling_steps: int = 0 + current_latent: torch.Tensor | None = None + current_image: Image.Image | None = None + current_image_sampling_step: int = 0 + id_live_preview: int = 0 + textinfo: str | None = None + time_start: float | None = None + need_restart: bool = False + server_start: float | None = None + + @dataclass + class OptionInfo: + default: Any = None + label: str = "" + component: Any = None + component_args: Callable[[], dict] | dict[str, Any] | None = None + onchange: Callable[[], None] | None = None + section: tuple[str, str] | None = None + refresh: Callable[[], None] | None = None + + class Option: + data_labels: dict[str, OptionInfo] + + def __init__(self): + self.data: dict[str, Any] = {} + + def add_option(self, key: str, info: OptionInfo): + pass + + def __getattr__(self, item: str): + if self.data is not None and item in self.data: + return self.data[item] + + if item in self.data_labels: + return self.data_labels[item].default + + return super().__getattribute__(item) + + opts = Option() + cmd_opts = argparse.Namespace() + state = State() + +else: + from modules.shared import OptionInfo, cmd_opts, opts, state