init

extensions/adetailer/CHANGELOG.md

@@ -0,0 +1,235 @@
+# Changelog
+
+## 2023-07-07
+
+- v23.7.4
+- batch count > 1일때 프롬프트의 인덱스 문제 수정
+
+- v23.7.5
+- i2i의 `cached_uc`와 `cached_c`가 p의 `cached_uc`와 `cached_c`가 다른 인스턴스가 되도록 수정
+
+## 2023-07-05
+
+- v23.7.3
+- 버그 수정
+  - `object()`가 json 직렬화 안되는 문제
+  - `process`를 호출함에 따라 배치 카운트가 2이상일 때, all_prompts가 고정되는 문제
+  - `ad-before`와 `ad-preview` 이미지 파일명이 실제 파일명과 다른 문제
+  - pydantic 2.0 호환성 문제
+
+## 2023-07-04
+
+- v23.7.2
+- `mediapipe_face_mesh_eyes_only` 모델 추가: `mediapipe_face_mesh`로 감지한 뒤 눈만 사용함.
+- 매 배치 시작 전에 `scripts.postprocess`를, 후에 `scripts.process`를 호출함.
+  - 컨트롤넷을 사용하면 소요 시간이 조금 늘어나지만 몇몇 문제 해결에 도움이 됨.
+- `lora_block_weight`를 스크립트 화이트리스트에 추가함.
+  - 한번이라도 ADetailer를 사용한 사람은 수동으로 추가해야함.
+
+## 2023-07-03
+
+- v23.7.1
+- `process_images`를 진행한 뒤 `StableDiffusionProcessing` 오브젝트의 close를 호출함
+- api 호출로 사용했는지 확인하는 속성 추가
+- `NansException`이 발생했을 때 중지하지 않고 남은 과정 계속 진행함
+
+## 2023-07-02
+
+- v23.7.0
+- `NansException`이 발생하면 로그에 표시하고 원본 이미지를 반환하게 설정
+- `rich`를 사용한 에러 트레이싱
+  - install.py에 `rich` 추가
+- 생성 중에 컴포넌트의 값을 변경하면 args의 값도 함께 변경되는 문제 수정 (issue #180)
+- 터미널 로그로 ad_prompt와 ad_negative_prompt에 적용된 실제 프롬프트 확인할 수 있음 (입력과 다를 경우에만)
+
+## 2023-06-28
+
+- v23.6.4
+- 최대 모델 수 5 -> 10개
+- ad_prompt와 ad_negative_prompt에 빈칸으로 놔두면 입력 프롬프트가 사용된다는 문구 추가
+- huggingface 모델 다운로드 실패시 로깅
+- 1st 모델이 `None`일 경우 나머지 입력을 무시하던 문제 수정
+- `--use-cpu` 에 `adetailer` 입력 시 cpu로 yolo모델을 사용함
+
+## 2023-06-20
+
+- v23.6.3
+- 컨트롤넷 inpaint 모델에 대해, 3가지 모듈을 사용할 수 있도록 함
+- Noise Multiplier 옵션 추가 (PR #149)
+- pydantic 최소 버전 1.10.8로 설정 (Issue #146)
+
+## 2023-06-05
+
+- v23.6.2
+- xyz_grid에서 ADetailer를 사용할 수 있게함.
+  - 8가지 옵션만 1st 탭에 적용되도록 함.
+
+## 2023-06-01
+
+- v23.6.1
+- `inpaint, scribble, lineart, openpose, tile` 5가지 컨트롤넷 모델 지원 (PR #107)
+- controlnet guidance start, end 인자 추가 (PR #107)
+- `modules.extensions`를 사용하여 컨트롤넷 확장을 불러오고 경로를 알아내로록 변경
+- ui에서 컨트롤넷을 별도 함수로 분리
+
+## 2023-05-30
+
+- v23.6.0
+- 스크립트의 이름을 `After Detailer`에서 `ADetailer`로 변경
+  - API 사용자는 변경 필요함
+- 몇몇 설정 변경
+  - `ad_conf` → `ad_confidence`. 0~100 사이의 int → 0.0~1.0 사이의 float
+  - `ad_inpaint_full_res` → `ad_inpaint_only_masked`
+  - `ad_inpaint_full_res_padding` → `ad_inpaint_only_masked_padding`
+- mediapipe face mesh 모델 추가
+  - mediapipe 최소 버전 `0.10.0`
+
+- rich traceback 제거함
+- huggingface 다운로드 실패할 때 에러가 나지 않게 하고 해당 모델을 제거함
+
+## 2023-05-26
+
+- v23.5.19
+- 1번째 탭에도 `None` 옵션을 추가함
+- api로 ad controlnet model에 inpaint가 아닌 다른 컨트롤넷 모델을 사용하지 못하도록 막음
+- adetailer 진행중에 total tqdm 진행바 업데이트를 멈춤
+- state.inturrupted 상태에서 adetailer 과정을 중지함
+- 컨트롤넷 process를 각 batch가 끝난 순간에만 호출하도록 변경
+
+### 2023-05-25
+
+- v23.5.18
+- 컨트롤넷 관련 수정
+  - unit의 `input_mode`를 `SIMPLE`로 모두 변경
+  - 컨트롤넷 유넷 훅과 하이잭 함수들을 adetailer를 실행할 때에만 되돌리는 기능 추가
+  - adetailer 처리가 끝난 뒤 컨트롤넷 스크립트의 process를 다시 진행함. (batch count 2 이상일때의 문제 해결)
+- 기본 활성 스크립트 목록에서 컨트롤넷을 뺌
+
+### 2023-05-22
+
+- v23.5.17
+- 컨트롤넷 확장이 있으면 컨트롤넷 스크립트를 활성화함. (컨트롤넷 관련 문제 해결)
+- 모든 컴포넌트에 elem_id 설정
+- ui에 버전을 표시함
+
+
+### 2023-05-19
+
+- v23.5.16
+- 추가한 옵션
+  - Mask min/max ratio
+  - Mask merge mode
+  - Restore faces after ADetailer
+- 옵션들을 Accordion으로 묶음
+
+### 2023-05-18
+
+- v23.5.15
+- 필요한 것만 임포트하도록 변경 (vae 로딩 오류 없어짐. 로딩 속도 빨라짐)
+
+### 2023-05-17
+
+- v23.5.14
+- `[SKIP]`으로 ad prompt 일부를 건너뛰는 기능 추가
+- bbox 정렬 옵션 추가
+- sd_webui 타입힌트를 만들어냄
+- enable checker와 관련된 api 오류 수정?
+
+### 2023-05-15
+
+- v23.5.13
+- `[SEP]`으로 ad prompt를 분리하여 적용하는 기능 추가
+- enable checker를 다시 pydantic으로 변경함
+- ui 관련 함수를 adetailer.ui 폴더로 분리함
+- controlnet을 사용할 때 모든 controlnet unit 비활성화
+- adetailer 폴더가 없으면 만들게 함
+
+### 2023-05-13
+
+- v23.5.12
+- `ad_enable`을 제외한 입력이 dict타입으로 들어오도록 변경
+  - web api로 사용할 때에 특히 사용하기 쉬움
+  - web api breaking change
+- `mask_preprocess` 인자를 넣지 않았던 오류 수정 (PR #47)
+- huggingface에서 모델을 다운로드하지 않는 옵션 추가 `--ad-no-huggingface`
+
+### 2023-05-12
+
+- v23.5.11
+- `ultralytics` 알람 제거
+- 필요없는 exif 인자 더 제거함
+- `use separate steps` 옵션 추가
+- ui 배치를 조정함
+
+### 2023-05-09
+
+- v23.5.10
+- 선택한 스크립트만 ADetailer에 적용하는 옵션 추가, 기본값 `True`. 설정 탭에서 지정가능.
+  - 기본값: `dynamic_prompting,dynamic_thresholding,wildcards,wildcard_recursive`
+- `person_yolov8s-seg.pt` 모델 추가
+- `ultralytics`의 최소 버전을 `8.0.97`로 설정 (C:\\ 문제 해결된 버전)
+
+### 2023-05-08
+
+- v23.5.9
+- 2가지 이상의 모델을 사용할 수 있음. 기본값: 2, 최대: 5
+- segment 모델을 사용할 수 있게 함. `person_yolov8n-seg.pt` 추가
+
+### 2023-05-07
+
+- v23.5.8
+- 프롬프트와 네거티브 프롬프트에 방향키 지원 (PR #24)
+- `mask_preprocess`를 추가함. 이전 버전과 시드값이 달라질 가능성 있음!
+- 이미지 처리가 일어났을 때에만 before이미지를 저장함
+- 설정창의 레이블을 ADetailer 대신 더 적절하게 수정함
+
+### 2023-05-06
+
+- v23.5.7
+- `ad_use_cfg_scale` 옵션 추가. cfg 스케일을 따로 사용할지 말지 결정함.
+- `ad_enable` 기본값을 `True`에서 `False`로 변경
+- `ad_model`의 기본값을 `None`에서 첫번째 모델로 변경
+- 최소 2개의 입력(ad_enable, ad_model)만 들어오면 작동하게 변경.
+
+- v23.5.7.post0
+- `init_controlnet_ext`을 controlnet_exists == True일때에만 실행
+- webui를 C드라이브 바로 밑에 설치한 사람들에게 `ultralytics` 경고 표시
+
+### 2023-05-05 (어린이날)
+
+- v23.5.5
+- `Save images before ADetailer` 옵션 추가
+- 입력으로 들어온 인자와 ALL_ARGS의 길이가 다르면 에러메세지
+- README.md에 설치방법 추가
+
+- v23.5.6
+- get_args에서 IndexError가 발생하면 자세한 에러메세지를 볼 수 있음
+- AdetailerArgs에 extra_params 내장
+- scripts_args를 딥카피함
+- postprocess_image를 약간 분리함
+
+- v23.5.6.post0
+- `init_controlnet_ext`에서 에러메세지를 자세히 볼 수 있음
+
+### 2023-05-04
+
+- v23.5.4
+- use pydantic for arguments validation
+- revert: ad_model to `None` as default
+- revert: `__future__` imports
+- lazily import yolo and mediapipe
+
+### 2023-05-03
+
+- v23.5.3.post0
+- remove `__future__` imports
+- change to copy scripts and scripts args
+
+- v23.5.3.post1
+- change default ad_model from `None`
+
+### 2023-05-02
+
+- v23.5.3
+- Remove `None` from model list and add `Enable ADetailer` checkbox.
+- install.py `skip_install` fix.

extensions/adetailer/LICENSE.md

@@ -0,0 +1,662 @@
+
+                    GNU AFFERO GENERAL PUBLIC LICENSE
+                       Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+  A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate.  Many developers of free software are heartened and
+encouraged by the resulting cooperation.  However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+  The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community.  It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server.  Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+  An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals.  This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU Affero General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Remote Network Interaction; Use with the GNU General Public License.
+
+  Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software.  This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time.  Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published
+    by the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source.  For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code.  There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.

extensions/adetailer/README.md

@@ -0,0 +1,105 @@
+# !After Detailer
+
+!After Detailer is a extension for stable diffusion webui, similar to Detection Detailer, except it uses ultralytics instead of the mmdet.
+
+## Install
+
+(from Mikubill/sd-webui-controlnet)
+
+1. Open "Extensions" tab.
+2. Open "Install from URL" tab in the tab.
+3. Enter `https://github.com/Bing-su/adetailer.git` to "URL for extension's git repository".
+4. Press "Install" button.
+5. Wait 5 seconds, and you will see the message "Installed into stable-diffusion-webui\extensions\adetailer. Use Installed tab to restart".
+6. Go to "Installed" tab, click "Check for updates", and then click "Apply and restart UI". (The next time you can also use this method to update extensions.)
+7. Completely restart A1111 webui including your terminal. (If you do not know what is a "terminal", you can reboot your computer: turn your computer off and turn it on again.)
+
+You can now install it directly from the Extensions tab.
+
+![image](https://i.imgur.com/g6GdRBT.png)
+
+You **DON'T** need to download any model from huggingface.
+
+## Options
+
+| Model, Prompts                    |                                       |                                                   |
+| --------------------------------- | ------------------------------------- | ------------------------------------------------- |
+| ADetailer model                   | Determine what to detect.             | `None` = disable                                  |
+| ADetailer prompt, negative prompt | Prompts and negative prompts to apply | If left blank, it will use the same as the input. |
+
+| Detection                            |                                                                                              |     |
+| ------------------------------------ | -------------------------------------------------------------------------------------------- | --- |
+| Detection model confidence threshold | Only objects with a detection model confidence above this threshold are used for inpainting. |     |
+| Mask min/max ratio                   | Only use masks whose area is between those ratios for the area of the entire image.          |     |
+
+If you want to exclude objects in the background, try setting the min ratio to around `0.01`.
+
+| Mask Preprocessing              |                                                                                                                                     |                                                                                         |
+| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
+| Mask x, y offset                | Moves the mask horizontally and vertically by                                                                                       |                                                                                         |
+| Mask erosion (-) / dilation (+) | Enlarge or reduce the detected mask.                                                                                                | [opencv example](https://docs.opencv.org/4.7.0/db/df6/tutorial_erosion_dilatation.html) |
+| Mask merge mode                 | `None`: Inpaint each mask<br/>`Merge`: Merge all masks and inpaint<br/>`Merge and Invert`: Merge all masks and Invert, then inpaint |                                                                                         |
+
+Applied in this order: x, y offset → erosion/dilation → merge/invert.
+
+#### Inpainting
+
+![image](https://i.imgur.com/wyWlT1n.png)
+
+Each option corresponds to a corresponding option on the inpaint tab.
+
+## ControlNet Inpainting
+
+You can use the ControlNet extension if you have ControlNet installed and ControlNet models.
+
+Support `inpaint, scribble, lineart, openpose, tile` controlnet models. Once you choose a model, the preprocessor is set automatically.
+
+## Model
+
+| Model                 | Target                | mAP 50                        | mAP 50-95                     |
+| --------------------- | --------------------- | ----------------------------- | ----------------------------- |
+| face_yolov8n.pt       | 2D / realistic face   | 0.660                         | 0.366                         |
+| face_yolov8s.pt       | 2D / realistic face   | 0.713                         | 0.404                         |
+| hand_yolov8n.pt       | 2D / realistic hand   | 0.767                         | 0.505                         |
+| person_yolov8n-seg.pt | 2D / realistic person | 0.782 (bbox)<br/>0.761 (mask) | 0.555 (bbox)<br/>0.460 (mask) |
+| person_yolov8s-seg.pt | 2D / realistic person | 0.824 (bbox)<br/>0.809 (mask) | 0.605 (bbox)<br/>0.508 (mask) |
+| mediapipe_face_full   | realistic face        | -                             | -                             |
+| mediapipe_face_short  | realistic face        | -                             | -                             |
+| mediapipe_face_mesh   | realistic face        | -                             | -                             |
+
+The yolo models can be found on huggingface [Bingsu/adetailer](https://huggingface.co/Bingsu/adetailer).
+
+### User Model
+
+Put your [ultralytics](https://github.com/ultralytics/ultralytics) model in `webui/models/adetailer`. The model name should end with `.pt` or `.pth`.
+
+It must be a bbox detection or segment model and use all label.
+
+### Dataset
+
+Datasets used for training the yolo models are:
+
+#### Face
+
+- [Anime Face CreateML](https://universe.roboflow.com/my-workspace-mph8o/anime-face-createml)
+- [xml2txt](https://universe.roboflow.com/0oooooo0/xml2txt-njqx1)
+- [AN](https://universe.roboflow.com/sed-b8vkf/an-lfg5i)
+- [wider face](http://shuoyang1213.me/WIDERFACE/index.html)
+
+#### Hand
+
+- [AnHDet](https://universe.roboflow.com/1-yshhi/anhdet)
+- [hand-detection-fuao9](https://universe.roboflow.com/catwithawand/hand-detection-fuao9)
+
+#### Person
+
+- [coco2017](https://cocodataset.org/#home) (only person)
+- [AniSeg](https://github.com/jerryli27/AniSeg)
+- [skytnt/anime-segmentation](https://huggingface.co/datasets/skytnt/anime-segmentation)
+
+## Example
+
+![image](https://i.imgur.com/38RSxSO.png)
+![image](https://i.imgur.com/2CYgjLx.png)
+
+[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/F1F1L7V2N)

extensions/adetailer/adetailer/__init__.py

@@ -0,0 +1,20 @@
+from .__version__ import __version__
+from .args import AD_ENABLE, ALL_ARGS, ADetailerArgs, EnableChecker
+from .common import PredictOutput, get_models
+from .mediapipe import mediapipe_predict
+from .ultralytics import ultralytics_predict
+
+AFTER_DETAILER = "ADetailer"
+
+__all__ = [
+    "__version__",
+    "AD_ENABLE",
+    "ADetailerArgs",
+    "AFTER_DETAILER",
+    "ALL_ARGS",
+    "EnableChecker",
+    "PredictOutput",
+    "get_models",
+    "mediapipe_predict",
+    "ultralytics_predict",
+]

extensions/adetailer/adetailer/__version__.py

@@ -0,0 +1 @@
+__version__ = "23.7.5"

extensions/adetailer/adetailer/args.py

@@ -0,0 +1,214 @@
+from __future__ import annotations
+
+from collections import UserList
+from functools import cached_property, partial
+from typing import Any, Literal, NamedTuple, Optional, Union
+
+import pydantic
+from pydantic import (
+    BaseModel,
+    Extra,
+    NonNegativeFloat,
+    NonNegativeInt,
+    PositiveInt,
+    confloat,
+    constr,
+    root_validator,
+    validator,
+)
+
+cn_model_regex = r".*(inpaint|tile|scribble|lineart|openpose).*|^None$"
+
+
+class Arg(NamedTuple):
+    attr: str
+    name: str
+
+
+class ArgsList(UserList):
+    @cached_property
+    def attrs(self) -> tuple[str]:
+        return tuple(attr for attr, _ in self)
+
+    @cached_property
+    def names(self) -> tuple[str]:
+        return tuple(name for _, name in self)
+
+
+class ADetailerArgs(BaseModel, extra=Extra.forbid):
+    ad_model: str = "None"
+    ad_prompt: str = ""
+    ad_negative_prompt: str = ""
+    ad_confidence: confloat(ge=0.0, le=1.0) = 0.3
+    ad_mask_min_ratio: confloat(ge=0.0, le=1.0) = 0.0
+    ad_mask_max_ratio: confloat(ge=0.0, le=1.0) = 1.0
+    ad_dilate_erode: int = 4
+    ad_x_offset: int = 0
+    ad_y_offset: int = 0
+    ad_mask_merge_invert: Literal["None", "Merge", "Merge and Invert"] = "None"
+    ad_mask_blur: NonNegativeInt = 4
+    ad_denoising_strength: confloat(ge=0.0, le=1.0) = 0.4
+    ad_inpaint_only_masked: bool = True
+    ad_inpaint_only_masked_padding: NonNegativeInt = 32
+    ad_use_inpaint_width_height: bool = False
+    ad_inpaint_width: PositiveInt = 512
+    ad_inpaint_height: PositiveInt = 512
+    ad_use_steps: bool = False
+    ad_steps: PositiveInt = 28
+    ad_use_cfg_scale: bool = False
+    ad_cfg_scale: NonNegativeFloat = 7.0
+    ad_use_noise_multiplier: bool = False
+    ad_noise_multiplier: confloat(ge=0.5, le=1.5) = 1.0
+    ad_restore_face: bool = False
+    ad_controlnet_model: constr(regex=cn_model_regex) = "None"
+    ad_controlnet_module: Optional[constr(regex=r".*inpaint.*|^None$")] = None
+    ad_controlnet_weight: confloat(ge=0.0, le=1.0) = 1.0
+    ad_controlnet_guidance_start: confloat(ge=0.0, le=1.0) = 0.0
+    ad_controlnet_guidance_end: confloat(ge=0.0, le=1.0) = 1.0
+    is_api: bool = True
+
+    @root_validator(skip_on_failure=True)
+    def ad_controlnt_module_validator(cls, values):  # noqa: N805
+        cn_model = values.get("ad_controlnet_model", "None")
+        cn_module = values.get("ad_controlnet_module", None)
+        if "inpaint" not in cn_model or cn_module == "None":
+            values["ad_controlnet_module"] = None
+        return values
+
+    @validator("is_api", pre=True)
+    def is_api_validator(cls, v: Any):  # noqa: N805
+        "tuple is json serializable but cannot be made with json deserialize."
+        return type(v) is not tuple
+
+    @staticmethod
+    def ppop(
+        p: dict[str, Any],
+        key: str,
+        pops: list[str] | None = None,
+        cond: Any = None,
+    ) -> None:
+        if pops is None:
+            pops = [key]
+        if key not in p:
+            return
+        value = p[key]
+        cond = (not bool(value)) if cond is None else value == cond
+
+        if cond:
+            for k in pops:
+                p.pop(k, None)
+
+    def extra_params(self, suffix: str = "") -> dict[str, Any]:
+        if self.ad_model == "None":
+            return {}
+
+        p = {name: getattr(self, attr) for attr, name in ALL_ARGS}
+        ppop = partial(self.ppop, p)
+
+        ppop("ADetailer prompt")
+        ppop("ADetailer negative prompt")
+        ppop("ADetailer mask min ratio", cond=0.0)
+        ppop("ADetailer mask max ratio", cond=1.0)
+        ppop("ADetailer x offset", cond=0)
+        ppop("ADetailer y offset", cond=0)
+        ppop("ADetailer mask merge/invert", cond="None")
+        ppop("ADetailer inpaint only masked", ["ADetailer inpaint padding"])
+        ppop(
+            "ADetailer use inpaint width/height",
+            [
+                "ADetailer use inpaint width/height",
+                "ADetailer inpaint width",
+                "ADetailer inpaint height",
+            ],
+        )
+        ppop(
+            "ADetailer use separate steps",
+            ["ADetailer use separate steps", "ADetailer steps"],
+        )
+        ppop(
+            "ADetailer use separate CFG scale",
+            ["ADetailer use separate CFG scale", "ADetailer CFG scale"],
+        )
+        ppop(
+            "ADetailer use separate noise multiplier",
+            ["ADetailer use separate noise multiplier", "ADetailer noise multiplier"],
+        )
+
+        ppop("ADetailer restore face")
+        ppop(
+            "ADetailer ControlNet model",
+            [
+                "ADetailer ControlNet model",
+                "ADetailer ControlNet module",
+                "ADetailer ControlNet weight",
+                "ADetailer ControlNet guidance start",
+                "ADetailer ControlNet guidance end",
+            ],
+            cond="None",
+        )
+        ppop("ADetailer ControlNet module")
+        ppop("ADetailer ControlNet weight", cond=1.0)
+        ppop("ADetailer ControlNet guidance start", cond=0.0)
+        ppop("ADetailer ControlNet guidance end", cond=1.0)
+
+        if suffix:
+            p = {k + suffix: v for k, v in p.items()}
+
+        return p
+
+
+class EnableChecker(BaseModel):
+    enable: bool
+    arg_list: list
+
+    def is_enabled(self) -> bool:
+        ad_model = ALL_ARGS[0].attr
+        if not self.enable:
+            return False
+        return any(arg.get(ad_model, "None") != "None" for arg in self.arg_list)
+
+
+_all_args = [
+    ("ad_enable", "ADetailer enable"),
+    ("ad_model", "ADetailer model"),
+    ("ad_prompt", "ADetailer prompt"),
+    ("ad_negative_prompt", "ADetailer negative prompt"),
+    ("ad_confidence", "ADetailer confidence"),
+    ("ad_mask_min_ratio", "ADetailer mask min ratio"),
+    ("ad_mask_max_ratio", "ADetailer mask max ratio"),
+    ("ad_x_offset", "ADetailer x offset"),
+    ("ad_y_offset", "ADetailer y offset"),
+    ("ad_dilate_erode", "ADetailer dilate/erode"),
+    ("ad_mask_merge_invert", "ADetailer mask merge/invert"),
+    ("ad_mask_blur", "ADetailer mask blur"),
+    ("ad_denoising_strength", "ADetailer denoising strength"),
+    ("ad_inpaint_only_masked", "ADetailer inpaint only masked"),
+    ("ad_inpaint_only_masked_padding", "ADetailer inpaint padding"),
+    ("ad_use_inpaint_width_height", "ADetailer use inpaint width/height"),
+    ("ad_inpaint_width", "ADetailer inpaint width"),
+    ("ad_inpaint_height", "ADetailer inpaint height"),
+    ("ad_use_steps", "ADetailer use separate steps"),
+    ("ad_steps", "ADetailer steps"),
+    ("ad_use_cfg_scale", "ADetailer use separate CFG scale"),
+    ("ad_cfg_scale", "ADetailer CFG scale"),
+    ("ad_use_noise_multiplier", "ADetailer use separate noise multiplier"),
+    ("ad_noise_multiplier", "ADetailer noise multiplier"),
+    ("ad_restore_face", "ADetailer restore face"),
+    ("ad_controlnet_model", "ADetailer ControlNet model"),
+    ("ad_controlnet_module", "ADetailer ControlNet module"),
+    ("ad_controlnet_weight", "ADetailer ControlNet weight"),
+    ("ad_controlnet_guidance_start", "ADetailer ControlNet guidance start"),
+    ("ad_controlnet_guidance_end", "ADetailer ControlNet guidance end"),
+]
+
+AD_ENABLE = Arg(*_all_args[0])
+_args = [Arg(*args) for args in _all_args[1:]]
+ALL_ARGS = ArgsList(_args)
+
+BBOX_SORTBY = [
+    "None",
+    "Position (left to right)",
+    "Position (center to edge)",
+    "Area (large to small)",
+]
+MASK_MERGE_INVERT = ["None", "Merge", "Merge and Invert"]

extensions/adetailer/adetailer/common.py

@@ -0,0 +1,127 @@
+from __future__ import annotations
+
+from collections import OrderedDict
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Optional, Union
+
+from huggingface_hub import hf_hub_download
+from PIL import Image, ImageDraw
+from rich import print
+
+repo_id = "Bingsu/adetailer"
+
+
+@dataclass
+class PredictOutput:
+    bboxes: list[list[int | float]] = field(default_factory=list)
+    masks: list[Image.Image] = field(default_factory=list)
+    preview: Optional[Image.Image] = None
+
+
+def hf_download(file: str):
+    try:
+        path = hf_hub_download(repo_id, file)
+    except Exception:
+        msg = f"[-] ADetailer: Failed to load model {file!r} from huggingface"
+        print(msg)
+        path = "INVALID"
+    return path
+
+
+def get_models(
+    model_dir: Union[str, Path], huggingface: bool = True
+) -> OrderedDict[str, Optional[str]]:
+    model_dir = Path(model_dir)
+    if model_dir.is_dir():
+        model_paths = [
+            p
+            for p in model_dir.rglob("*")
+            if p.is_file() and p.suffix in (".pt", ".pth")
+        ]
+    else:
+        model_paths = []
+
+    models = OrderedDict()
+    if huggingface:
+        models.update(
+            {
+                "face_yolov8n.pt": hf_download("face_yolov8n.pt"),
+                "face_yolov8s.pt": hf_download("face_yolov8s.pt"),
+                "hand_yolov8n.pt": hf_download("hand_yolov8n.pt"),
+                "person_yolov8n-seg.pt": hf_download("person_yolov8n-seg.pt"),
+                "person_yolov8s-seg.pt": hf_download("person_yolov8s-seg.pt"),
+            }
+        )
+    models.update(
+        {
+            "mediapipe_face_full": None,
+            "mediapipe_face_short": None,
+            "mediapipe_face_mesh": None,
+            "mediapipe_face_mesh_eyes_only": None,
+        }
+    )
+
+    invalid_keys = [k for k, v in models.items() if v == "INVALID"]
+    for key in invalid_keys:
+        models.pop(key)
+
+    for path in model_paths:
+        if path.name in models:
+            continue
+        models[path.name] = str(path)
+
+    return models
+
+
+def create_mask_from_bbox(
+    bboxes: list[list[float]], shape: tuple[int, int]
+) -> list[Image.Image]:
+    """
+    Parameters
+    ----------
+        bboxes: list[list[float]]
+            list of [x1, y1, x2, y2]
+            bounding boxes
+        shape: tuple[int, int]
+            shape of the image (width, height)
+
+    Returns
+    -------
+        masks: list[Image.Image]
+        A list of masks
+
+    """
+    masks = []
+    for bbox in bboxes:
+        mask = Image.new("L", shape, 0)
+        mask_draw = ImageDraw.Draw(mask)
+        mask_draw.rectangle(bbox, fill=255)
+        masks.append(mask)
+    return masks
+
+
+def create_bbox_from_mask(
+    masks: list[Image.Image], shape: tuple[int, int]
+) -> list[list[int]]:
+    """
+    Parameters
+    ----------
+        masks: list[Image.Image]
+            A list of masks
+        shape: tuple[int, int]
+            shape of the image (width, height)
+
+    Returns
+    -------
+        bboxes: list[list[float]]
+        A list of bounding boxes
+
+    """
+    bboxes = []
+    for mask in masks:
+        mask = mask.resize(shape)
+        bbox = mask.getbbox()
+        if bbox is not None:
+            bboxes.append(list(bbox))
+    return bboxes

extensions/adetailer/adetailer/mask.py

@@ -0,0 +1,245 @@
+from __future__ import annotations
+
+from enum import IntEnum
+from functools import partial, reduce
+from math import dist
+
+import cv2
+import numpy as np
+from PIL import Image, ImageChops
+
+from adetailer.args import MASK_MERGE_INVERT
+from adetailer.common import PredictOutput
+
+
+class SortBy(IntEnum):
+    NONE = 0
+    LEFT_TO_RIGHT = 1
+    CENTER_TO_EDGE = 2
+    AREA = 3
+
+
+class MergeInvert(IntEnum):
+    NONE = 0
+    MERGE = 1
+    MERGE_INVERT = 2
+
+
+def _dilate(arr: np.ndarray, value: int) -> np.ndarray:
+    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (value, value))
+    return cv2.dilate(arr, kernel, iterations=1)
+
+
+def _erode(arr: np.ndarray, value: int) -> np.ndarray:
+    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (value, value))
+    return cv2.erode(arr, kernel, iterations=1)
+
+
+def dilate_erode(img: Image.Image, value: int) -> Image.Image:
+    """
+    The dilate_erode function takes an image and a value.
+    If the value is positive, it dilates the image by that amount.
+    If the value is negative, it erodes the image by that amount.
+
+    Parameters
+    ----------
+        img: PIL.Image.Image
+            the image to be processed
+        value: int
+            kernel size of dilation or erosion
+
+    Returns
+    -------
+        PIL.Image.Image
+            The image that has been dilated or eroded
+    """
+    if value == 0:
+        return img
+
+    arr = np.array(img)
+    arr = _dilate(arr, value) if value > 0 else _erode(arr, -value)
+
+    return Image.fromarray(arr)
+
+
+def offset(img: Image.Image, x: int = 0, y: int = 0) -> Image.Image:
+    """
+    The offset function takes an image and offsets it by a given x(→) and y(↑) value.
+
+    Parameters
+    ----------
+        mask: Image.Image
+            Pass the mask image to the function
+        x: int
+            →
+        y: int
+            ↑
+
+    Returns
+    -------
+        PIL.Image.Image
+            A new image that is offset by x and y
+    """
+    return ImageChops.offset(img, x, -y)
+
+
+def is_all_black(img: Image.Image) -> bool:
+    arr = np.array(img)
+    return cv2.countNonZero(arr) == 0
+
+
+def bbox_area(bbox: list[float]):
+    return (bbox[2] - bbox[0]) * (bbox[3] - bbox[1])
+
+
+def mask_preprocess(
+    masks: list[Image.Image],
+    kernel: int = 0,
+    x_offset: int = 0,
+    y_offset: int = 0,
+    merge_invert: int | MergeInvert | str = MergeInvert.NONE,
+) -> list[Image.Image]:
+    """
+    The mask_preprocess function takes a list of masks and preprocesses them.
+    It dilates and erodes the masks, and offsets them by x_offset and y_offset.
+
+    Parameters
+    ----------
+        masks: list[Image.Image]
+            A list of masks
+        kernel: int
+            kernel size of dilation or erosion
+        x_offset: int
+            →
+        y_offset: int
+            ↑
+
+    Returns
+    -------
+        list[Image.Image]
+            A list of processed masks
+    """
+    if not masks:
+        return []
+
+    if x_offset != 0 or y_offset != 0:
+        masks = [offset(m, x_offset, y_offset) for m in masks]
+
+    if kernel != 0:
+        masks = [dilate_erode(m, kernel) for m in masks]
+        masks = [m for m in masks if not is_all_black(m)]
+
+    return mask_merge_invert(masks, mode=merge_invert)
+
+
+# Bbox sorting
+def _key_left_to_right(bbox: list[float]) -> float:
+    """
+    Left to right
+
+    Parameters
+    ----------
+    bbox: list[float]
+        list of [x1, y1, x2, y2]
+    """
+    return bbox[0]
+
+
+def _key_center_to_edge(bbox: list[float], *, center: tuple[float, float]) -> float:
+    """
+    Center to edge
+
+    Parameters
+    ----------
+    bbox: list[float]
+        list of [x1, y1, x2, y2]
+    image: Image.Image
+        the image
+    """
+    bbox_center = ((bbox[0] + bbox[2]) / 2, (bbox[1] + bbox[3]) / 2)
+    return dist(center, bbox_center)
+
+
+def _key_area(bbox: list[float]) -> float:
+    """
+    Large to small
+
+    Parameters
+    ----------
+    bbox: list[float]
+        list of [x1, y1, x2, y2]
+    """
+    return -bbox_area(bbox)
+
+
+def sort_bboxes(
+    pred: PredictOutput, order: int | SortBy = SortBy.NONE
+) -> PredictOutput:
+    if order == SortBy.NONE or len(pred.bboxes) <= 1:
+        return pred
+
+    if order == SortBy.LEFT_TO_RIGHT:
+        key = _key_left_to_right
+    elif order == SortBy.CENTER_TO_EDGE:
+        width, height = pred.preview.size
+        center = (width / 2, height / 2)
+        key = partial(_key_center_to_edge, center=center)
+    elif order == SortBy.AREA:
+        key = _key_area
+    else:
+        raise RuntimeError
+
+    items = len(pred.bboxes)
+    idx = sorted(range(items), key=lambda i: key(pred.bboxes[i]))
+    pred.bboxes = [pred.bboxes[i] for i in idx]
+    pred.masks = [pred.masks[i] for i in idx]
+    return pred
+
+
+# Filter by ratio
+def is_in_ratio(bbox: list[float], low: float, high: float, orig_area: int) -> bool:
+    area = bbox_area(bbox)
+    return low <= area / orig_area <= high
+
+
+def filter_by_ratio(pred: PredictOutput, low: float, high: float) -> PredictOutput:
+    if not pred.bboxes:
+        return pred
+
+    w, h = pred.preview.size
+    orig_area = w * h
+    items = len(pred.bboxes)
+    idx = [i for i in range(items) if is_in_ratio(pred.bboxes[i], low, high, orig_area)]
+    pred.bboxes = [pred.bboxes[i] for i in idx]
+    pred.masks = [pred.masks[i] for i in idx]
+    return pred
+
+
+# Merge / Invert
+def mask_merge(masks: list[Image.Image]) -> list[Image.Image]:
+    arrs = [np.array(m) for m in masks]
+    arr = reduce(cv2.bitwise_or, arrs)
+    return [Image.fromarray(arr)]
+
+
+def mask_invert(masks: list[Image.Image]) -> list[Image.Image]:
+    return [ImageChops.invert(m) for m in masks]
+
+
+def mask_merge_invert(
+    masks: list[Image.Image], mode: int | MergeInvert | str
+) -> list[Image.Image]:
+    if isinstance(mode, str):
+        mode = MASK_MERGE_INVERT.index(mode)
+
+    if mode == MergeInvert.NONE or not masks:
+        return masks
+
+    if mode == MergeInvert.MERGE:
+        return mask_merge(masks)
+
+    if mode == MergeInvert.MERGE_INVERT:
+        merged = mask_merge(masks)
+        return mask_invert(merged)
+
+    raise RuntimeError

extensions/adetailer/adetailer/mediapipe.py

@@ -0,0 +1,184 @@
+from __future__ import annotations
+
+from functools import partial
+
+import numpy as np
+from PIL import Image, ImageDraw
+
+from adetailer import PredictOutput
+from adetailer.common import create_bbox_from_mask, create_mask_from_bbox
+
+
+def mediapipe_predict(
+    model_type: str, image: Image.Image, confidence: float = 0.3
+) -> PredictOutput:
+    mapping = {
+        "mediapipe_face_short": partial(mediapipe_face_detection, 0),
+        "mediapipe_face_full": partial(mediapipe_face_detection, 1),
+        "mediapipe_face_mesh": mediapipe_face_mesh,
+        "mediapipe_face_mesh_eyes_only": mediapipe_face_mesh_eyes_only,
+    }
+    if model_type in mapping:
+        func = mapping[model_type]
+        return func(image, confidence)
+    msg = f"[-] ADetailer: Invalid mediapipe model type: {model_type}, Available: {list(mapping.keys())!r}"
+    raise RuntimeError(msg)
+
+
+def mediapipe_face_detection(
+    model_type: int, image: Image.Image, confidence: float = 0.3
+) -> PredictOutput:
+    import mediapipe as mp
+
+    img_width, img_height = image.size
+
+    mp_face_detection = mp.solutions.face_detection
+    draw_util = mp.solutions.drawing_utils
+
+    img_array = np.array(image)
+
+    with mp_face_detection.FaceDetection(
+        model_selection=model_type, min_detection_confidence=confidence
+    ) as face_detector:
+        pred = face_detector.process(img_array)
+
+    if pred.detections is None:
+        return PredictOutput()
+
+    preview_array = img_array.copy()
+
+    bboxes = []
+    for detection in pred.detections:
+        draw_util.draw_detection(preview_array, detection)
+
+        bbox = detection.location_data.relative_bounding_box
+        x1 = bbox.xmin * img_width
+        y1 = bbox.ymin * img_height
+        w = bbox.width * img_width
+        h = bbox.height * img_height
+        x2 = x1 + w
+        y2 = y1 + h
+
+        bboxes.append([x1, y1, x2, y2])
+
+    masks = create_mask_from_bbox(bboxes, image.size)
+    preview = Image.fromarray(preview_array)
+
+    return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
+
+
+def get_convexhull(points: np.ndarray) -> list[tuple[int, int]]:
+    """
+    Parameters
+    ----------
+      points: An ndarray of shape (n, 2) containing the 2D points.
+
+    Returns
+    -------
+      list[tuple[int, int]]: Input for the draw.polygon function
+    """
+    from scipy.spatial import ConvexHull
+
+    hull = ConvexHull(points)
+    vertices = hull.vertices
+    return list(zip(points[vertices, 0], points[vertices, 1]))
+
+
+def mediapipe_face_mesh(image: Image.Image, confidence: float = 0.3) -> PredictOutput:
+    import mediapipe as mp
+
+    mp_face_mesh = mp.solutions.face_mesh
+    draw_util = mp.solutions.drawing_utils
+    drawing_styles = mp.solutions.drawing_styles
+
+    w, h = image.size
+
+    with mp_face_mesh.FaceMesh(
+        static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
+    ) as face_mesh:
+        arr = np.array(image)
+        pred = face_mesh.process(arr)
+
+        if pred.multi_face_landmarks is None:
+            return PredictOutput()
+
+        preview = arr.copy()
+        masks = []
+
+        for landmarks in pred.multi_face_landmarks:
+            draw_util.draw_landmarks(
+                image=preview,
+                landmark_list=landmarks,
+                connections=mp_face_mesh.FACEMESH_TESSELATION,
+                landmark_drawing_spec=None,
+                connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
+            )
+
+            points = np.array([(land.x * w, land.y * h) for land in landmarks.landmark])
+            outline = get_convexhull(points)
+
+            mask = Image.new("L", image.size, "black")
+            draw = ImageDraw.Draw(mask)
+            draw.polygon(outline, fill="white")
+            masks.append(mask)
+
+        bboxes = create_bbox_from_mask(masks, image.size)
+        preview = Image.fromarray(preview)
+        return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
+
+
+def mediapipe_face_mesh_eyes_only(
+    image: Image.Image, confidence: float = 0.3
+) -> PredictOutput:
+    import mediapipe as mp
+
+    mp_face_mesh = mp.solutions.face_mesh
+
+    left_idx = np.array(list(mp_face_mesh.FACEMESH_LEFT_EYE)).flatten()
+    right_idx = np.array(list(mp_face_mesh.FACEMESH_RIGHT_EYE)).flatten()
+
+    w, h = image.size
+
+    with mp_face_mesh.FaceMesh(
+        static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
+    ) as face_mesh:
+        arr = np.array(image)
+        pred = face_mesh.process(arr)
+
+        if pred.multi_face_landmarks is None:
+            return PredictOutput()
+
+        preview = image.copy()
+        masks = []
+
+        for landmarks in pred.multi_face_landmarks:
+            points = np.array([(land.x * w, land.y * h) for land in landmarks.landmark])
+            left_eyes = points[left_idx]
+            right_eyes = points[right_idx]
+            left_outline = get_convexhull(left_eyes)
+            right_outline = get_convexhull(right_eyes)
+
+            mask = Image.new("L", image.size, "black")
+            draw = ImageDraw.Draw(mask)
+            for outline in (left_outline, right_outline):
+                draw.polygon(outline, fill="white")
+            masks.append(mask)
+
+        bboxes = create_bbox_from_mask(masks, image.size)
+        preview = draw_preview(preview, bboxes, masks)
+        return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
+
+
+def draw_preview(
+    preview: Image.Image, bboxes: list[list[int]], masks: list[Image.Image]
+) -> Image.Image:
+    red = Image.new("RGB", preview.size, "red")
+    for mask in masks:
+        masked = Image.composite(red, preview, mask)
+        preview = Image.blend(preview, masked, 0.25)
+
+    draw = ImageDraw.Draw(preview)
+    for bbox in bboxes:
+        draw.rectangle(bbox, outline="red", width=2)
+
+    return preview

extensions/adetailer/adetailer/traceback.py

@@ -0,0 +1,158 @@
+from __future__ import annotations
+
+import io
+import platform
+import sys
+from typing import Any, Callable
+
+from rich.console import Console, Group
+from rich.panel import Panel
+from rich.table import Table
+from rich.traceback import Traceback
+
+from adetailer.__version__ import __version__
+
+
+def processing(*args: Any) -> dict[str, Any]:
+    try:
+        from modules.processing import (
+            StableDiffusionProcessingImg2Img,
+            StableDiffusionProcessingTxt2Img,
+        )
+    except ImportError:
+        return {}
+
+    p = None
+    for arg in args:
+        if isinstance(
+            arg, (StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img)
+        ):
+            p = arg
+            break
+
+    if p is None:
+        return {}
+
+    info = {
+        "prompt": p.prompt,
+        "negative_prompt": p.negative_prompt,
+        "n_iter": p.n_iter,
+        "batch_size": p.batch_size,
+        "width": p.width,
+        "height": p.height,
+        "sampler_name": p.sampler_name,
+        "enable_hr": getattr(p, "enable_hr", False),
+        "hr_upscaler": getattr(p, "hr_upscaler", ""),
+    }
+
+    info.update(sd_models())
+    return info
+
+
+def sd_models() -> dict[str, str]:
+    try:
+        from modules import shared
+
+        opts = shared.opts
+    except Exception:
+        return {}
+
+    return {
+        "checkpoint": getattr(opts, "sd_model_checkpoint", "------"),
+        "vae": getattr(opts, "sd_vae", "------"),
+        "unet": getattr(opts, "sd_unet", "------"),
+    }
+
+
+def ad_args(*args: Any) -> dict[str, Any]:
+    ad_args = [
+        arg
+        for arg in args
+        if isinstance(arg, dict) and arg.get("ad_model", "None") != "None"
+    ]
+    if not ad_args:
+        return {}
+
+    arg0 = ad_args[0]
+    is_api = arg0.get("is_api", True)
+    return {
+        "version": __version__,
+        "ad_model": arg0["ad_model"],
+        "ad_prompt": arg0.get("ad_prompt", ""),
+        "ad_negative_prompt": arg0.get("ad_negative_prompt", ""),
+        "ad_controlnet_model": arg0.get("ad_controlnet_model", "None"),
+        "is_api": type(is_api) is not tuple,
+    }
+
+
+def sys_info() -> dict[str, Any]:
+    try:
+        import launch
+
+        version = launch.git_tag()
+        commit = launch.commit_hash()
+    except Exception:
+        version = commit = "------"
+
+    return {
+        "Platform": platform.platform(),
+        "Python": sys.version,
+        "Version": version,
+        "Commit": commit,
+        "Commandline": sys.argv,
+    }
+
+
+def get_table(title: str, data: dict[str, Any]) -> Table:
+    table = Table(title=title, highlight=True)
+    table.add_column(" ", justify="right", style="dim")
+    table.add_column("Value")
+    for key, value in data.items():
+        if not isinstance(value, str):
+            value = repr(value)
+        table.add_row(key, value)
+
+    return table
+
+
+def force_terminal_value():
+    try:
+        from modules.shared import cmd_opts
+
+        return True if hasattr(cmd_opts, "skip_torch_cuda_test") else None
+    except Exception:
+        return None
+
+
+def rich_traceback(func: Callable) -> Callable:
+    force_terminal = force_terminal_value()
+
+    def wrapper(*args, **kwargs):
+        string = io.StringIO()
+        width = Console().width
+        width = width - 4 if width > 4 else None
+        console = Console(file=string, force_terminal=force_terminal, width=width)
+        try:
+            return func(*args, **kwargs)
+        except Exception as e:
+            tables = [
+                get_table(title, data)
+                for title, data in [
+                    ("System info", sys_info()),
+                    ("Inputs", processing(*args)),
+                    ("ADetailer", ad_args(*args)),
+                ]
+                if data
+            ]
+            tables.append(Traceback())
+
+            console.print(Panel(Group(*tables)))
+            output = "\n" + string.getvalue()
+
+            try:
+                error = e.__class__(output)
+            except Exception:
+                error = RuntimeError(output)
+            raise error from None
+
+    return wrapper

extensions/adetailer/adetailer/ui.py

@@ -0,0 +1,505 @@
+from __future__ import annotations
+
+from functools import partial
+from types import SimpleNamespace
+from typing import Any
+
+import gradio as gr
+
+from adetailer import AFTER_DETAILER, __version__
+from adetailer.args import AD_ENABLE, ALL_ARGS, MASK_MERGE_INVERT
+from controlnet_ext import controlnet_exists, get_cn_models
+
+cn_module_choices = [
+    "inpaint_global_harmonious",
+    "inpaint_only",
+    "inpaint_only+lama",
+]
+
+
+class Widgets(SimpleNamespace):
+    def tolist(self):
+        return [getattr(self, attr) for attr in ALL_ARGS.attrs]
+
+
+def gr_interactive(value: bool = True):
+    return gr.update(interactive=value)
+
+
+def ordinal(n: int) -> str:
+    d = {1: "st", 2: "nd", 3: "rd"}
+    return str(n) + ("th" if 11 <= n % 100 <= 13 else d.get(n % 10, "th"))
+
+
+def suffix(n: int, c: str = " ") -> str:
+    return "" if n == 0 else c + ordinal(n + 1)
+
+
+def on_widget_change(state: dict, value: Any, *, attr: str):
+    state[attr] = value
+    return state
+
+
+def on_generate_click(state: dict, *values: Any):
+    for attr, value in zip(ALL_ARGS.attrs, values):
+        state[attr] = value
+    state["is_api"] = ()
+    return state
+
+
+def on_cn_model_update(cn_model: str):
+    if "inpaint" in cn_model:
+        return gr.update(
+            visible=True, choices=cn_module_choices, value=cn_module_choices[0]
+        )
+    return gr.update(visible=False, choices=["None"], value="None")
+
+
+def elem_id(item_id: str, n: int, is_img2img: bool) -> str:
+    tap = "img2img" if is_img2img else "txt2img"
+    suf = suffix(n, "_")
+    return f"script_{tap}_adetailer_{item_id}{suf}"
+
+
+def adui(
+    num_models: int,
+    is_img2img: bool,
+    model_list: list[str],
+    t2i_button: gr.Button,
+    i2i_button: gr.Button,
+):
+    states = []
+    infotext_fields = []
+    eid = partial(elem_id, n=0, is_img2img=is_img2img)
+
+    with gr.Accordion(AFTER_DETAILER, open=False, elem_id=eid("ad_main_accordion")):
+        with gr.Row():
+            with gr.Column(scale=6):
+                ad_enable = gr.Checkbox(
+                    label="Enable ADetailer",
+                    value=False,
+                    visible=True,
+                    elem_id=eid("ad_enable"),
+                )
+
+            with gr.Column(scale=1, min_width=180):
+                gr.Markdown(
+                    f"v{__version__}",
+                    elem_id=eid("ad_version"),
+                )
+
+        infotext_fields.append((ad_enable, AD_ENABLE.name))
+
+        with gr.Group(), gr.Tabs():
+            for n in range(num_models):
+                with gr.Tab(ordinal(n + 1)):
+                    state, infofields = one_ui_group(
+                        n=n,
+                        is_img2img=is_img2img,
+                        model_list=model_list,
+                        t2i_button=t2i_button,
+                        i2i_button=i2i_button,
+                    )
+
+                states.append(state)
+                infotext_fields.extend(infofields)
+
+    # components: [bool, dict, dict, ...]
+    components = [ad_enable, *states]
+    return components, infotext_fields
+
+
+def one_ui_group(
+    n: int,
+    is_img2img: bool,
+    model_list: list[str],
+    t2i_button: gr.Button,
+    i2i_button: gr.Button,
+):
+    w = Widgets()
+    state = gr.State({})
+    eid = partial(elem_id, n=n, is_img2img=is_img2img)
+
+    with gr.Row():
+        model_choices = [*model_list, "None"] if n == 0 else ["None", *model_list]
+
+        w.ad_model = gr.Dropdown(
+            label="ADetailer model" + suffix(n),
+            choices=model_choices,
+            value=model_choices[0],
+            visible=True,
+            type="value",
+            elem_id=eid("ad_model"),
+        )
+
+    with gr.Group():
+        with gr.Row(elem_id=eid("ad_toprow_prompt")):
+            w.ad_prompt = gr.Textbox(
+                label="ad_prompt" + suffix(n),
+                show_label=False,
+                lines=3,
+                placeholder="ADetailer prompt"
+                + suffix(n)
+                + "\nIf blank, the main prompt is used.",
+                elem_id=eid("ad_prompt"),
+            )
+
+        with gr.Row(elem_id=eid("ad_toprow_negative_prompt")):
+            w.ad_negative_prompt = gr.Textbox(
+                label="ad_negative_prompt" + suffix(n),
+                show_label=False,
+                lines=2,
+                placeholder="ADetailer negative prompt"
+                + suffix(n)
+                + "\nIf blank, the main negative prompt is used.",
+                elem_id=eid("ad_negative_prompt"),
+            )
+
+    with gr.Group():
+        with gr.Accordion(
+            "Detection", open=False, elem_id=eid("ad_detection_accordion")
+        ):
+            detection(w, n, is_img2img)
+
+        with gr.Accordion(
+            "Mask Preprocessing",
+            open=False,
+            elem_id=eid("ad_mask_preprocessing_accordion"),
+        ):
+            mask_preprocessing(w, n, is_img2img)
+
+        with gr.Accordion(
+            "Inpainting", open=False, elem_id=eid("ad_inpainting_accordion")
+        ):
+            inpainting(w, n, is_img2img)
+
+    with gr.Group():
+        controlnet(w, n, is_img2img)
+
+    all_inputs = [state, *w.tolist()]
+    target_button = i2i_button if is_img2img else t2i_button
+    target_button.click(
+        fn=on_generate_click, inputs=all_inputs, outputs=state, queue=False
+    )
+
+    infotext_fields = [(getattr(w, attr), name + suffix(n)) for attr, name in ALL_ARGS]
+
+    return state, infotext_fields
+
+
+def detection(w: Widgets, n: int, is_img2img: bool):
+    eid = partial(elem_id, n=n, is_img2img=is_img2img)
+
+    with gr.Row():
+        with gr.Column():
+            w.ad_confidence = gr.Slider(
+                label="Detection model confidence threshold" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                value=0.3,
+                visible=True,
+                elem_id=eid("ad_confidence"),
+            )
+
+        with gr.Column(variant="compact"):
+            w.ad_mask_min_ratio = gr.Slider(
+                label="Mask min area ratio" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.001,
+                value=0.0,
+                visible=True,
+                elem_id=eid("ad_mask_min_ratio"),
+            )
+            w.ad_mask_max_ratio = gr.Slider(
+                label="Mask max area ratio" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.001,
+                value=1.0,
+                visible=True,
+                elem_id=eid("ad_mask_max_ratio"),
+            )
+
+
+def mask_preprocessing(w: Widgets, n: int, is_img2img: bool):
+    eid = partial(elem_id, n=n, is_img2img=is_img2img)
+
+    with gr.Group():
+        with gr.Row():
+            with gr.Column(variant="compact"):
+                w.ad_x_offset = gr.Slider(
+                    label="Mask x(→) offset" + suffix(n),
+                    minimum=-200,
+                    maximum=200,
+                    step=1,
+                    value=0,
+                    visible=True,
+                    elem_id=eid("ad_x_offset"),
+                )
+                w.ad_y_offset = gr.Slider(
+                    label="Mask y(↑) offset" + suffix(n),
+                    minimum=-200,
+                    maximum=200,
+                    step=1,
+                    value=0,
+                    visible=True,
+                    elem_id=eid("ad_y_offset"),
+                )
+
+            with gr.Column(variant="compact"):
+                w.ad_dilate_erode = gr.Slider(
+                    label="Mask erosion (-) / dilation (+)" + suffix(n),
+                    minimum=-128,
+                    maximum=128,
+                    step=4,
+                    value=4,
+                    visible=True,
+                    elem_id=eid("ad_dilate_erode"),
+                )
+
+        with gr.Row():
+            w.ad_mask_merge_invert = gr.Radio(
+                label="Mask merge mode" + suffix(n),
+                choices=MASK_MERGE_INVERT,
+                value="None",
+                elem_id=eid("ad_mask_merge_invert"),
+            )
+
+
+def inpainting(w: Widgets, n: int, is_img2img: bool):
+    eid = partial(elem_id, n=n, is_img2img=is_img2img)
+
+    with gr.Group():
+        with gr.Row():
+            w.ad_mask_blur = gr.Slider(
+                label="Inpaint mask blur" + suffix(n),
+                minimum=0,
+                maximum=64,
+                step=1,
+                value=4,
+                visible=True,
+                elem_id=eid("ad_mask_blur"),
+            )
+
+            w.ad_denoising_strength = gr.Slider(
+                label="Inpaint denoising strength" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                value=0.4,
+                visible=True,
+                elem_id=eid("ad_denoising_strength"),
+            )
+
+        with gr.Row():
+            with gr.Column(variant="compact"):
+                w.ad_inpaint_only_masked = gr.Checkbox(
+                    label="Inpaint only masked" + suffix(n),
+                    value=True,
+                    visible=True,
+                    elem_id=eid("ad_inpaint_only_masked"),
+                )
+                w.ad_inpaint_only_masked_padding = gr.Slider(
+                    label="Inpaint only masked padding, pixels" + suffix(n),
+                    minimum=0,
+                    maximum=256,
+                    step=4,
+                    value=32,
+                    visible=True,
+                    elem_id=eid("ad_inpaint_only_masked_padding"),
+                )
+
+                w.ad_inpaint_only_masked.change(
+                    gr_interactive,
+                    inputs=w.ad_inpaint_only_masked,
+                    outputs=w.ad_inpaint_only_masked_padding,
+                    queue=False,
+                )
+
+            with gr.Column(variant="compact"):
+                w.ad_use_inpaint_width_height = gr.Checkbox(
+                    label="Use separate width/height" + suffix(n),
+                    value=False,
+                    visible=True,
+                    elem_id=eid("ad_use_inpaint_width_height"),
+                )
+
+                w.ad_inpaint_width = gr.Slider(
+                    label="inpaint width" + suffix(n),
+                    minimum=64,
+                    maximum=2048,
+                    step=4,
+                    value=512,
+                    visible=True,
+                    elem_id=eid("ad_inpaint_width"),
+                )
+
+                w.ad_inpaint_height = gr.Slider(
+                    label="inpaint height" + suffix(n),
+                    minimum=64,
+                    maximum=2048,
+                    step=4,
+                    value=512,
+                    visible=True,
+                    elem_id=eid("ad_inpaint_height"),
+                )
+
+                w.ad_use_inpaint_width_height.change(
+                    lambda value: (gr_interactive(value), gr_interactive(value)),
+                    inputs=w.ad_use_inpaint_width_height,
+                    outputs=[w.ad_inpaint_width, w.ad_inpaint_height],
+                    queue=False,
+                )
+
+        with gr.Row():
+            with gr.Column(variant="compact"):
+                w.ad_use_steps = gr.Checkbox(
+                    label="Use separate steps" + suffix(n),
+                    value=False,
+                    visible=True,
+                    elem_id=eid("ad_use_steps"),
+                )
+
+                w.ad_steps = gr.Slider(
+                    label="ADetailer steps" + suffix(n),
+                    minimum=1,
+                    maximum=150,
+                    step=1,
+                    value=28,
+                    visible=True,
+                    elem_id=eid("ad_steps"),
+                )
+
+                w.ad_use_steps.change(
+                    gr_interactive,
+                    inputs=w.ad_use_steps,
+                    outputs=w.ad_steps,
+                    queue=False,
+                )
+
+            with gr.Column(variant="compact"):
+                w.ad_use_cfg_scale = gr.Checkbox(
+                    label="Use separate CFG scale" + suffix(n),
+                    value=False,
+                    visible=True,
+                    elem_id=eid("ad_use_cfg_scale"),
+                )
+
+                w.ad_cfg_scale = gr.Slider(
+                    label="ADetailer CFG scale" + suffix(n),
+                    minimum=0.0,
+                    maximum=30.0,
+                    step=0.5,
+                    value=7.0,
+                    visible=True,
+                    elem_id=eid("ad_cfg_scale"),
+                )
+
+                w.ad_use_cfg_scale.change(
+                    gr_interactive,
+                    inputs=w.ad_use_cfg_scale,
+                    outputs=w.ad_cfg_scale,
+                    queue=False,
+                )
+
+        with gr.Row():
+            with gr.Column(variant="compact"):
+                w.ad_use_noise_multiplier = gr.Checkbox(
+                    label="Use separate noise multiplier" + suffix(n),
+                    value=False,
+                    visible=True,
+                    elem_id=eid("ad_use_noise_multiplier"),
+                )
+
+                w.ad_noise_multiplier = gr.Slider(
+                    label="Noise multiplier for img2img" + suffix(n),
+                    minimum=0.5,
+                    maximum=1.5,
+                    step=0.01,
+                    value=1.0,
+                    visible=True,
+                    elem_id=eid("ad_noise_multiplier"),
+                )
+
+                w.ad_use_noise_multiplier.change(
+                    gr_interactive,
+                    inputs=w.ad_use_noise_multiplier,
+                    outputs=w.ad_noise_multiplier,
+                    queue=False,
+                )
+
+            w.ad_restore_face = gr.Checkbox(
+                label="Restore faces after ADetailer" + suffix(n),
+                value=False,
+                elem_id=eid("ad_restore_face"),
+            )
+
+
+def controlnet(w: Widgets, n: int, is_img2img: bool):
+    eid = partial(elem_id, n=n, is_img2img=is_img2img)
+    cn_models = ["None", *get_cn_models()]
+
+    with gr.Row(variant="panel"):
+        with gr.Column(variant="compact"):
+            w.ad_controlnet_model = gr.Dropdown(
+                label="ControlNet model" + suffix(n),
+                choices=cn_models,
+                value="None",
+                visible=True,
+                type="value",
+                interactive=controlnet_exists,
+                elem_id=eid("ad_controlnet_model"),
+            )
+
+            w.ad_controlnet_module = gr.Dropdown(
+                label="ControlNet module" + suffix(n),
+                choices=cn_module_choices,
+                value="inpaint_global_harmonious",
+                visible=False,
+                type="value",
+                interactive=controlnet_exists,
+                elem_id=eid("ad_controlnet_module"),
+            )
+
+            w.ad_controlnet_weight = gr.Slider(
+                label="ControlNet weight" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                value=1.0,
+                visible=True,
+                interactive=controlnet_exists,
+                elem_id=eid("ad_controlnet_weight"),
+            )
+
+            w.ad_controlnet_model.change(
+                on_cn_model_update,
+                inputs=w.ad_controlnet_model,
+                outputs=w.ad_controlnet_module,
+                queue=False,
+            )
+
+        with gr.Column(variant="compact"):
+            w.ad_controlnet_guidance_start = gr.Slider(
+                label="ControlNet guidance start" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                value=0.0,
+                visible=True,
+                interactive=controlnet_exists,
+                elem_id=eid("ad_controlnet_guidance_start"),
+            )
+
+            w.ad_controlnet_guidance_end = gr.Slider(
+                label="ControlNet guidance end" + suffix(n),
+                minimum=0.0,
+                maximum=1.0,
+                step=0.01,
+                value=1.0,
+                visible=True,
+                interactive=controlnet_exists,
+                elem_id=eid("ad_controlnet_guidance_end"),
+            )

extensions/adetailer/adetailer/ultralytics.py

@@ -0,0 +1,54 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import cv2
+from PIL import Image
+
+from adetailer import PredictOutput
+from adetailer.common import create_mask_from_bbox
+
+
+def ultralytics_predict(
+    model_path: str | Path,
+    image: Image.Image,
+    confidence: float = 0.3,
+    device: str = "",
+) -> PredictOutput:
+    from ultralytics import YOLO
+
+    model_path = str(model_path)
+
+    model = YOLO(model_path)
+    pred = model(image, conf=confidence, device=device)
+
+    bboxes = pred[0].boxes.xyxy.cpu().numpy()
+    if bboxes.size == 0:
+        return PredictOutput()
+    bboxes = bboxes.tolist()
+
+    if pred[0].masks is None:
+        masks = create_mask_from_bbox(bboxes, image.size)
+    else:
+        masks = mask_to_pil(pred[0].masks.data, image.size)
+    preview = pred[0].plot()
+    preview = cv2.cvtColor(preview, cv2.COLOR_BGR2RGB)
+    preview = Image.fromarray(preview)
+
+    return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
+
+
+def mask_to_pil(masks, shape: tuple[int, int]) -> list[Image.Image]:
+    """
+    Parameters
+    ----------
+    masks: torch.Tensor, dtype=torch.float32, shape=(N, H, W).
+        The device can be CUDA, but `to_pil_image` takes care of that.
+
+    shape: tuple[int, int]
+        (width, height) of the original image
+    """
+    from torchvision.transforms.functional import to_pil_image
+
+    n = masks.shape[0]
+    return [to_pil_image(masks[i], mode="L").resize(shape) for i in range(n)]

extensions/adetailer/controlnet_ext/__init__.py

@@ -0,0 +1,7 @@
+from .controlnet_ext import ControlNetExt, controlnet_exists, get_cn_models
+
+__all__ = [
+    "ControlNetExt",
+    "controlnet_exists",
+    "get_cn_models",
+]

extensions/adetailer/controlnet_ext/controlnet_ext.py

@@ -0,0 +1,140 @@
+from __future__ import annotations
+
+import importlib
+import re
+from functools import lru_cache
+from pathlib import Path
+
+from modules import extensions, sd_models, shared
+from modules.paths import data_path, models_path, script_path
+
+ext_path = Path(data_path, "extensions")
+ext_builtin_path = Path(script_path, "extensions-builtin")
+controlnet_exists = False
+controlnet_path = None
+cn_base_path = ""
+
+for extension in extensions.active():
+    if not extension.enabled:
+        continue
+    # For cases like sd-webui-controlnet-master
+    if "sd-webui-controlnet" in extension.name:
+        controlnet_exists = True
+        controlnet_path = Path(extension.path)
+        cn_base_path = ".".join(controlnet_path.parts[-2:])
+        break
+
+cn_model_module = {
+    "inpaint": "inpaint_global_harmonious",
+    "scribble": "t2ia_sketch_pidi",
+    "lineart": "lineart_coarse",
+    "openpose": "openpose_full",
+    "tile": None,
+}
+cn_model_regex = re.compile("|".join(cn_model_module.keys()))
+
+
+class ControlNetExt:
+    def __init__(self):
+        self.cn_models = ["None"]
+        self.cn_available = False
+        self.external_cn = None
+
+    def init_controlnet(self):
+        import_path = cn_base_path + ".scripts.external_code"
+
+        self.external_cn = importlib.import_module(import_path, "external_code")
+        self.cn_available = True
+        models = self.external_cn.get_models()
+        self.cn_models.extend(m for m in models if cn_model_regex.search(m))
+
+    def update_scripts_args(
+        self,
+        p,
+        model: str,
+        module: str | None,
+        weight: float,
+        guidance_start: float,
+        guidance_end: float,
+    ):
+        if (not self.cn_available) or model == "None":
+            return
+
+        if module is None:
+            for m, v in cn_model_module.items():
+                if m in model:
+                    module = v
+                    break
+
+        cn_units = [
+            self.external_cn.ControlNetUnit(
+                model=model,
+                weight=weight,
+                control_mode=self.external_cn.ControlMode.BALANCED,
+                module=module,
+                guidance_start=guidance_start,
+                guidance_end=guidance_end,
+                pixel_perfect=True,
+            )
+        ]
+
+        self.external_cn.update_cn_script_in_processing(p, cn_units)
+
+
+def get_cn_model_dirs() -> list[Path]:
+    cn_model_dir = Path(models_path, "ControlNet")
+    if controlnet_path is not None:
+        cn_model_dir_old = controlnet_path.joinpath("models")
+    else:
+        cn_model_dir_old = None
+    ext_dir1 = shared.opts.data.get("control_net_models_path", "")
+    ext_dir2 = shared.opts.data.get("controlnet_dir", "")
+
+    dirs = [cn_model_dir]
+    for ext_dir in [cn_model_dir_old, ext_dir1, ext_dir2]:
+        if ext_dir:
+            dirs.append(Path(ext_dir))
+
+    return dirs
+
+
+@lru_cache
+def _get_cn_models() -> list[str]:
+    """
+    Since we can't import ControlNet, we use a function that does something like
+    controlnet's `list(global_state.cn_models_names.values())`.
+    """
+    cn_model_exts = (".pt", ".pth", ".ckpt", ".safetensors")
+    dirs = get_cn_model_dirs()
+    name_filter = shared.opts.data.get("control_net_models_name_filter", "")
+    name_filter = name_filter.strip(" ").lower()
+
+    model_paths = []
+
+    for base in dirs:
+        if not base.exists():
+            continue
+
+        for p in base.rglob("*"):
+            if (
+                p.is_file()
+                and p.suffix in cn_model_exts
+                and cn_model_regex.search(p.name)
+            ):
+                if name_filter and name_filter not in p.name.lower():
+                    continue
+                model_paths.append(p)
+    model_paths.sort(key=lambda p: p.name)
+
+    models = []
+    for p in model_paths:
+        model_hash = sd_models.model_hash(p)
+        name = f"{p.stem} [{model_hash}]"
+        models.append(name)
+    return models
+
+
+def get_cn_models() -> list[str]:
+    if controlnet_exists:
+        return _get_cn_models()
+    return []

extensions/adetailer/controlnet_ext/restore.py

@@ -0,0 +1,49 @@
+from __future__ import annotations
+
+from contextlib import contextmanager
+
+from modules import img2img, processing, shared
+
+
+def cn_restore_unet_hook(p, cn_latest_network):
+    if cn_latest_network is not None:
+        unet = p.sd_model.model.diffusion_model
+        cn_latest_network.restore(unet)
+
+
+class CNHijackRestore:
+    def __init__(self):
+        self.process = hasattr(processing, "__controlnet_original_process_images_inner")
+        self.img2img = hasattr(img2img, "__controlnet_original_process_batch")
+
+    def __enter__(self):
+        if self.process:
+            self.orig_process = processing.process_images_inner
+            processing.process_images_inner = getattr(
+                processing, "__controlnet_original_process_images_inner"
+            )
+        if self.img2img:
+            self.orig_img2img = img2img.process_batch
+            img2img.process_batch = getattr(
+                img2img, "__controlnet_original_process_batch"
+            )
+
+    def __exit__(self, *args, **kwargs):
+        if self.process:
+            processing.process_images_inner = self.orig_process
+        if self.img2img:
+            img2img.process_batch = self.orig_img2img
+
+
+@contextmanager
+def cn_allow_script_control():
+    orig = False
+    if "control_net_allow_script_control" in shared.opts.data:
+        try:
+            orig = shared.opts.data["control_net_allow_script_control"]
+            shared.opts.data["control_net_allow_script_control"] = True
+            yield
+        finally:
+            shared.opts.data["control_net_allow_script_control"] = orig
+    else:
+        yield

extensions/adetailer/install.py

@@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import importlib.util
+import subprocess
+import sys
+from importlib.metadata import version  # python >= 3.8
+
+from packaging.version import parse
+
+
+def is_installed(
+    package: str, min_version: str | None = None, max_version: str | None = None
+):
+    try:
+        spec = importlib.util.find_spec(package)
+    except ModuleNotFoundError:
+        return False
+
+    if spec is None:
+        return False
+
+    if not min_version and not max_version:
+        return True
+
+    if not min_version:
+        min_version = "0.0.0"
+    if not max_version:
+        max_version = "99999999.99999999.99999999"
+
+    if package == "google.protobuf":
+        package = "protobuf"
+
+    try:
+        pkg_version = version(package)
+        return parse(min_version) <= parse(pkg_version) <= parse(max_version)
+    except Exception:
+        return False
+
+
+def run_pip(*args):
+    subprocess.run([sys.executable, "-m", "pip", "install", *args])
+
+
+def install():
+    deps = [
+        # requirements
+        ("ultralytics", "8.0.97", None),
+        ("mediapipe", "0.10.0", None),
+        ("huggingface_hub", None, None),
+        ("pydantic", "1.10.8", None),
+        ("rich", "13.4.2", None),
+        # mediapipe
+        ("protobuf", "3.20.0", "3.20.9999"),
+    ]
+
+    for pkg, low, high in deps:
+        # https://github.com/protocolbuffers/protobuf/tree/main/python
+        name = "google.protobuf" if pkg == "protobuf" else pkg
+
+        if not is_installed(name, low, high):
+            if low and high:
+                cmd = f"{pkg}>={low},<={high}"
+            elif low:
+                cmd = f"{pkg}>={low}"
+            elif high:
+                cmd = f"{pkg}<={high}"
+            else:
+                cmd = pkg
+
+            run_pip("-U", cmd)
+
+
+try:
+    import launch
+
+    skip_install = launch.args.skip_install
+except Exception:
+    skip_install = False
+
+if not skip_install:
+    install()

extensions/adetailer/preload.py

@@ -0,0 +1,9 @@
+import argparse
+
+
+def preload(parser: argparse.ArgumentParser):
+    parser.add_argument(
+        "--ad-no-huggingface",
+        action="store_true",
+        help="Don't use adetailer models from huggingface",
+    )

extensions/adetailer/pyproject.toml

@@ -0,0 +1,26 @@
+[project]
+name = "adetailer"
+description = "An object detection and auto-mask extension for stable diffusion webui."
+authors = [
+    {name = "dowon", email = "[email protected]"},
+]
+requires-python = ">=3.8,<3.12"
+readme = "README.md"
+license = {text = "AGPL-3.0"}
+
+[project.urls]
+repository = "https://github.com/Bing-su/adetailer"
+
+[tool.isort]
+profile = "black"
+known_first_party = ["launch", "modules"]
+
+[tool.ruff]
+select = ["A", "B", "C4", "C90", "E", "EM", "F", "FA", "I001", "ISC", "N", "PIE", "PT", "RET", "RUF", "SIM", "UP", "W"]
+ignore = ["B008", "B905", "E501", "F401", "UP007"]
+
+[tool.ruff.isort]
+known-first-party = ["launch", "modules"]
+
+[tool.ruff.per-file-ignores]
+"sd_webui/*.py" = ["B027", "F403"]

extensions/adetailer/scripts/!adetailer.py

@@ -0,0 +1,784 @@
+from __future__ import annotations
+
+import os
+import platform
+import re
+import sys
+import traceback
+from contextlib import contextmanager
+from copy import copy, deepcopy
+from functools import partial
+from pathlib import Path
+from textwrap import dedent
+from typing import Any
+
+import gradio as gr
+import torch
+from rich import print
+
+import modules
+from adetailer import (
+    AFTER_DETAILER,
+    __version__,
+    get_models,
+    mediapipe_predict,
+    ultralytics_predict,
+)
+from adetailer.args import ALL_ARGS, BBOX_SORTBY, ADetailerArgs, EnableChecker
+from adetailer.common import PredictOutput
+from adetailer.mask import filter_by_ratio, mask_preprocess, sort_bboxes
+from adetailer.traceback import rich_traceback
+from adetailer.ui import adui, ordinal, suffix
+from controlnet_ext import ControlNetExt, controlnet_exists, get_cn_models
+from controlnet_ext.restore import (
+    CNHijackRestore,
+    cn_allow_script_control,
+    cn_restore_unet_hook,
+)
+from sd_webui import images, safe, script_callbacks, scripts, shared
+from sd_webui.devices import NansException
+from sd_webui.paths import data_path, models_path
+from sd_webui.processing import (
+    Processed,
+    StableDiffusionProcessingImg2Img,
+    create_infotext,
+    process_images,
+)
+from sd_webui.shared import cmd_opts, opts, state
+
+no_huggingface = getattr(cmd_opts, "ad_no_huggingface", False)
+adetailer_dir = Path(models_path, "adetailer")
+model_mapping = get_models(adetailer_dir, huggingface=not no_huggingface)
+txt2img_submit_button = img2img_submit_button = None
+SCRIPT_DEFAULT = "dynamic_prompting,dynamic_thresholding,wildcard_recursive,wildcards,lora_block_weight"
+
+if (
+    not adetailer_dir.exists()
+    and adetailer_dir.parent.exists()
+    and os.access(adetailer_dir.parent, os.W_OK)
+):
+    adetailer_dir.mkdir()
+
+print(
+    f"[-] ADetailer initialized. version: {__version__}, num models: {len(model_mapping)}"
+)
+
+
+@contextmanager
+def change_torch_load():
+    orig = torch.load
+    try:
+        torch.load = safe.unsafe_torch_load
+        yield
+    finally:
+        torch.load = orig
+
+
+@contextmanager
+def pause_total_tqdm():
+    orig = opts.data.get("multiple_tqdm", True)
+    try:
+        opts.data["multiple_tqdm"] = False
+        yield
+    finally:
+        opts.data["multiple_tqdm"] = orig
+
+
+@contextmanager
+def preseve_prompts(p):
+    all_pt = copy(p.all_prompts)
+    all_ng = copy(p.all_negative_prompts)
+    try:
+        yield
+    finally:
+        p.all_prompts = all_pt
+        p.all_negative_prompts = all_ng
+
+
+class AfterDetailerScript(scripts.Script):
+    def __init__(self):
+        super().__init__()
+        self.ultralytics_device = self.get_ultralytics_device()
+
+        self.controlnet_ext = None
+        self.cn_script = None
+        self.cn_latest_network = None
+
+    def __repr__(self):
+        return f"{self.__class__.__name__}(version={__version__})"
+
+    def title(self):
+        return AFTER_DETAILER
+
+    def show(self, is_img2img):
+        return scripts.AlwaysVisible
+
+    def ui(self, is_img2img):
+        num_models = opts.data.get("ad_max_models", 2)
+        model_list = list(model_mapping.keys())
+
+        components, infotext_fields = adui(
+            num_models,
+            is_img2img,
+            model_list,
+            txt2img_submit_button,
+            img2img_submit_button,
+        )
+
+        self.infotext_fields = infotext_fields
+        return components
+
+    def init_controlnet_ext(self) -> None:
+        if self.controlnet_ext is not None:
+            return
+        self.controlnet_ext = ControlNetExt()
+
+        if controlnet_exists:
+            try:
+                self.controlnet_ext.init_controlnet()
+            except ImportError:
+                error = traceback.format_exc()
+                print(
+                    f"[-] ADetailer: ControlNetExt init failed:\n{error}",
+                    file=sys.stderr,
+                )
+
+    def update_controlnet_args(self, p, args: ADetailerArgs) -> None:
+        if self.controlnet_ext is None:
+            self.init_controlnet_ext()
+
+        if (
+            self.controlnet_ext is not None
+            and self.controlnet_ext.cn_available
+            and args.ad_controlnet_model != "None"
+        ):
+            self.controlnet_ext.update_scripts_args(
+                p,
+                model=args.ad_controlnet_model,
+                module=args.ad_controlnet_module,
+                weight=args.ad_controlnet_weight,
+                guidance_start=args.ad_controlnet_guidance_start,
+                guidance_end=args.ad_controlnet_guidance_end,
+            )
+
+    def is_ad_enabled(self, *args_) -> bool:
+        arg_list = [arg for arg in args_ if isinstance(arg, dict)]
+        if not args_ or not arg_list or not isinstance(args_[0], (bool, dict)):
+            message = f"""
+                       [-] ADetailer: Invalid arguments passed to ADetailer.
+                           input: {args_!r}
+                       """
+            raise ValueError(dedent(message))
+        enable = args_[0] if isinstance(args_[0], bool) else True
+        checker = EnableChecker(enable=enable, arg_list=arg_list)
+        return checker.is_enabled()
+
+    def get_args(self, p, *args_) -> list[ADetailerArgs]:
+        """
+        `args_` is at least 1 in length by `is_ad_enabled` immediately above
+        """
+        args = [arg for arg in args_ if isinstance(arg, dict)]
+
+        if not args:
+            message = f"[-] ADetailer: Invalid arguments passed to ADetailer: {args_!r}"
+            raise ValueError(message)
+
+        if hasattr(p, "adetailer_xyz"):
+            args[0].update(p.adetailer_xyz)
+
+        all_inputs = []
+
+        for n, arg_dict in enumerate(args, 1):
+            try:
+                inp = ADetailerArgs(**arg_dict)
+            except ValueError as e:
+                msgs = [
+                    f"[-] ADetailer: ValidationError when validating {ordinal(n)} arguments: {e}\n"
+                ]
+                for attr in ALL_ARGS.attrs:
+                    arg = arg_dict.get(attr)
+                    dtype = type(arg)
+                    arg = "DEFAULT" if arg is None else repr(arg)
+                    msgs.append(f"    {attr}: {arg} ({dtype})")
+                raise ValueError("\n".join(msgs)) from e
+
+            all_inputs.append(inp)
+
+        return all_inputs
+
+    def extra_params(self, arg_list: list[ADetailerArgs]) -> dict:
+        params = {}
+        for n, args in enumerate(arg_list):
+            params.update(args.extra_params(suffix=suffix(n)))
+        params["ADetailer version"] = __version__
+        return params
+
+    @staticmethod
+    def get_ultralytics_device() -> str:
+        if "adetailer" in shared.cmd_opts.use_cpu:
+            return "cpu"
+
+        if platform.system() == "Darwin":
+            return ""
+
+        if any(getattr(cmd_opts, vram, False) for vram in ["lowvram", "medvram"]):
+            return "cpu"
+
+        return ""
+
+    def prompt_blank_replacement(
+        self, all_prompts: list[str], i: int, default: str
+    ) -> str:
+        if not all_prompts:
+            return default
+        if i < len(all_prompts):
+            return all_prompts[i]
+        j = i % len(all_prompts)
+        return all_prompts[j]
+
+    def _get_prompt(
+        self, ad_prompt: str, all_prompts: list[str], i: int, default: str
+    ) -> list[str]:
+        prompts = re.split(r"\s*\[SEP\]\s*", ad_prompt)
+        blank_replacement = self.prompt_blank_replacement(all_prompts, i, default)
+        for n in range(len(prompts)):
+            if not prompts[n]:
+                prompts[n] = blank_replacement
+        return prompts
+
+    def get_prompt(self, p, args: ADetailerArgs) -> tuple[list[str], list[str]]:
+        i = p._ad_idx
+
+        prompt = self._get_prompt(args.ad_prompt, p.all_prompts, i, p.prompt)
+        negative_prompt = self._get_prompt(
+            args.ad_negative_prompt, p.all_negative_prompts, i, p.negative_prompt
+        )
+
+        return prompt, negative_prompt
+
+    def get_seed(self, p) -> tuple[int, int]:
+        i = p._ad_idx
+
+        if not p.all_seeds:
+            seed = p.seed
+        elif i < len(p.all_seeds):
+            seed = p.all_seeds[i]
+        else:
+            j = i % len(p.all_seeds)
+            seed = p.all_seeds[j]
+
+        if not p.all_subseeds:
+            subseed = p.subseed
+        elif i < len(p.all_subseeds):
+            subseed = p.all_subseeds[i]
+        else:
+            j = i % len(p.all_subseeds)
+            subseed = p.all_subseeds[j]
+
+        return seed, subseed
+
+    def get_width_height(self, p, args: ADetailerArgs) -> tuple[int, int]:
+        if args.ad_use_inpaint_width_height:
+            width = args.ad_inpaint_width
+            height = args.ad_inpaint_height
+        else:
+            width = p.width
+            height = p.height
+
+        return width, height
+
+    def get_steps(self, p, args: ADetailerArgs) -> int:
+        if args.ad_use_steps:
+            return args.ad_steps
+        return p.steps
+
+    def get_cfg_scale(self, p, args: ADetailerArgs) -> float:
+        if args.ad_use_cfg_scale:
+            return args.ad_cfg_scale
+        return p.cfg_scale
+
+    def get_initial_noise_multiplier(self, p, args: ADetailerArgs) -> float | None:
+        if args.ad_use_noise_multiplier:
+            return args.ad_noise_multiplier
+        return None
+
+    def infotext(self, p) -> str:
+        return create_infotext(
+            p, p.all_prompts, p.all_seeds, p.all_subseeds, None, 0, 0
+        )
+
+    def write_params_txt(self, p) -> None:
+        infotext = self.infotext(p)
+        params_txt = Path(data_path, "params.txt")
+        params_txt.write_text(infotext, encoding="utf-8")
+
+    def script_filter(self, p, args: ADetailerArgs):
+        script_runner = copy(p.scripts)
+        script_args = deepcopy(p.script_args)
+        self.disable_controlnet_units(script_args)
+
+        ad_only_seleted_scripts = opts.data.get("ad_only_seleted_scripts", True)
+        if not ad_only_seleted_scripts:
+            return script_runner, script_args
+
+        ad_script_names = opts.data.get("ad_script_names", SCRIPT_DEFAULT)
+        script_names_set = {
+            name
+            for script_name in ad_script_names.split(",")
+            for name in (script_name, script_name.strip())
+        }
+
+        if args.ad_controlnet_model != "None":
+            script_names_set.add("controlnet")
+
+        filtered_alwayson = []
+        for script_object in script_runner.alwayson_scripts:
+            filepath = script_object.filename
+            filename = Path(filepath).stem
+            if filename in script_names_set:
+                filtered_alwayson.append(script_object)
+            if filename == "controlnet":
+                self.cn_script = script_object
+                self.cn_latest_network = script_object.latest_network
+
+        script_runner.alwayson_scripts = filtered_alwayson
+        return script_runner, script_args
+
+    def disable_controlnet_units(self, script_args: list[Any]) -> None:
+        for obj in script_args:
+            if "controlnet" in obj.__class__.__name__.lower():
+                if hasattr(obj, "enabled"):
+                    obj.enabled = False
+                if hasattr(obj, "input_mode"):
+                    obj.input_mode = getattr(obj.input_mode, "SIMPLE", "simple")
+
+            elif isinstance(obj, dict) and "module" in obj:
+                obj["enabled"] = False
+
+    def get_i2i_p(self, p, args: ADetailerArgs, image):
+        seed, subseed = self.get_seed(p)
+        width, height = self.get_width_height(p, args)
+        steps = self.get_steps(p, args)
+        cfg_scale = self.get_cfg_scale(p, args)
+        initial_noise_multiplier = self.get_initial_noise_multiplier(p, args)
+
+        sampler_name = p.sampler_name
+        if sampler_name in ["PLMS", "UniPC"]:
+            sampler_name = "Euler"
+
+        i2i = StableDiffusionProcessingImg2Img(
+            init_images=[image],
+            resize_mode=0,
+            denoising_strength=args.ad_denoising_strength,
+            mask=None,
+            mask_blur=args.ad_mask_blur,
+            inpainting_fill=1,
+            inpaint_full_res=args.ad_inpaint_only_masked,
+            inpaint_full_res_padding=args.ad_inpaint_only_masked_padding,
+            inpainting_mask_invert=0,
+            initial_noise_multiplier=initial_noise_multiplier,
+            sd_model=p.sd_model,
+            outpath_samples=p.outpath_samples,
+            outpath_grids=p.outpath_grids,
+            prompt="",  # replace later
+            negative_prompt="",
+            styles=p.styles,
+            seed=seed,
+            subseed=subseed,
+            subseed_strength=p.subseed_strength,
+            seed_resize_from_h=p.seed_resize_from_h,
+            seed_resize_from_w=p.seed_resize_from_w,
+            sampler_name=sampler_name,
+            batch_size=1,
+            n_iter=1,
+            steps=steps,
+            cfg_scale=cfg_scale,
+            width=width,
+            height=height,
+            restore_faces=args.ad_restore_face,
+            tiling=p.tiling,
+            extra_generation_params=p.extra_generation_params,
+            do_not_save_samples=True,
+            do_not_save_grid=True,
+        )
+
+        i2i.cached_c = [None, None]
+        i2i.cached_uc = [None, None]
+        i2i.scripts, i2i.script_args = self.script_filter(p, args)
+        i2i._disable_adetailer = True
+
+        if args.ad_controlnet_model != "None":
+            self.update_controlnet_args(i2i, args)
+        else:
+            i2i.control_net_enabled = False
+
+        return i2i
+
+    def save_image(self, p, image, *, condition: str, suffix: str) -> None:
+        i = p._ad_idx
+        if p.all_prompts:
+            i %= len(p.all_prompts)
+            save_prompt = p.all_prompts[i]
+        else:
+            save_prompt = p.prompt
+        seed, _ = self.get_seed(p)
+
+        if opts.data.get(condition, False):
+            images.save_image(
+                image=image,
+                path=p.outpath_samples,
+                basename="",
+                seed=seed,
+                prompt=save_prompt,
+                extension=opts.samples_format,
+                info=self.infotext(p),
+                p=p,
+                suffix=suffix,
+            )
+
+    def get_ad_model(self, name: str):
+        if name not in model_mapping:
+            msg = f"[-] ADetailer: Model {name!r} not found. Available models: {list(model_mapping.keys())}"
+            raise ValueError(msg)
+        return model_mapping[name]
+
+    def sort_bboxes(self, pred: PredictOutput) -> PredictOutput:
+        sortby = opts.data.get("ad_bbox_sortby", BBOX_SORTBY[0])
+        sortby_idx = BBOX_SORTBY.index(sortby)
+        return sort_bboxes(pred, sortby_idx)
+
+    def pred_preprocessing(self, pred: PredictOutput, args: ADetailerArgs):
+        pred = filter_by_ratio(
+            pred, low=args.ad_mask_min_ratio, high=args.ad_mask_max_ratio
+        )
+        pred = self.sort_bboxes(pred)
+        return mask_preprocess(
+            pred.masks,
+            kernel=args.ad_dilate_erode,
+            x_offset=args.ad_x_offset,
+            y_offset=args.ad_y_offset,
+            merge_invert=args.ad_mask_merge_invert,
+        )
+
+    def i2i_prompts_replace(
+        self, i2i, prompts: list[str], negative_prompts: list[str], j: int
+    ) -> None:
+        i1 = min(j, len(prompts) - 1)
+        i2 = min(j, len(negative_prompts) - 1)
+        prompt = prompts[i1]
+        negative_prompt = negative_prompts[i2]
+        i2i.prompt = prompt
+        i2i.negative_prompt = negative_prompt
+
+    @staticmethod
+    def compare_prompt(p, processed, n: int = 0):
+        if p.prompt != processed.all_prompts[0]:
+            print(
+                f"[-] ADetailer: applied {ordinal(n + 1)} ad_prompt: {processed.all_prompts[0]!r}"
+            )
+
+        if p.negative_prompt != processed.all_negative_prompts[0]:
+            print(
+                f"[-] ADetailer: applied {ordinal(n + 1)} ad_negative_prompt: {processed.all_negative_prompts[0]!r}"
+            )
+
+    def need_call_process(self, p) -> bool:
+        i = p._ad_idx
+        bs = p.batch_size
+        return i % bs == bs - 1
+
+    def need_call_postprocess(self, p) -> bool:
+        i = p._ad_idx
+        bs = p.batch_size
+        return i % bs == 0
+
+    @rich_traceback
+    def process(self, p, *args_):
+        if getattr(p, "_disable_adetailer", False):
+            return
+
+        if self.is_ad_enabled(*args_):
+            arg_list = self.get_args(p, *args_)
+            extra_params = self.extra_params(arg_list)
+            p.extra_generation_params.update(extra_params)
+
+    def _postprocess_image(self, p, pp, args: ADetailerArgs, *, n: int = 0) -> bool:
+        """
+        Returns
+        -------
+            bool
+
+            `True` if image was processed, `False` otherwise.
+        """
+        if state.interrupted:
+            return False
+
+        i = p._ad_idx
+
+        i2i = self.get_i2i_p(p, args, pp.image)
+        seed, subseed = self.get_seed(p)
+        ad_prompts, ad_negatives = self.get_prompt(p, args)
+
+        is_mediapipe = args.ad_model.lower().startswith("mediapipe")
+
+        kwargs = {}
+        if is_mediapipe:
+            predictor = mediapipe_predict
+            ad_model = args.ad_model
+        else:
+            predictor = ultralytics_predict
+            ad_model = self.get_ad_model(args.ad_model)
+            kwargs["device"] = self.ultralytics_device
+
+        with change_torch_load():
+            pred = predictor(ad_model, pp.image, args.ad_confidence, **kwargs)
+
+        masks = self.pred_preprocessing(pred, args)
+
+        if not masks:
+            print(
+                f"[-] ADetailer: nothing detected on image {i + 1} with {ordinal(n + 1)} settings."
+            )
+            return False
+
+        self.save_image(
+            p,
+            pred.preview,
+            condition="ad_save_previews",
+            suffix="-ad-preview" + suffix(n, "-"),
+        )
+
+        steps = len(masks)
+        processed = None
+        state.job_count += steps
+
+        if is_mediapipe:
+            print(f"mediapipe: {steps} detected.")
+
+        _user_pt = p.prompt
+        _user_ng = p.negative_prompt
+
+        p2 = copy(i2i)
+        for j in range(steps):
+            p2.image_mask = masks[j]
+            self.i2i_prompts_replace(p2, ad_prompts, ad_negatives, j)
+
+            if re.match(r"^\s*\[SKIP\]\s*$", p2.prompt):
+                continue
+
+            p2.seed = seed + j
+            p2.subseed = subseed + j
+
+            try:
+                processed = process_images(p2)
+            except NansException as e:
+                msg = f"[-] ADetailer: 'NansException' occurred with {ordinal(n + 1)} settings.\n{e}"
+                print(msg, file=sys.stderr)
+                continue
+            finally:
+                p2.close()
+
+            self.compare_prompt(p2, processed, n=n)
+            p2 = copy(i2i)
+            p2.init_images = [processed.images[0]]
+
+        if processed is not None:
+            pp.image = processed.images[0]
+            return True
+
+        return False
+
+    @rich_traceback
+    def postprocess_image(self, p, pp, *args_):
+        if getattr(p, "_disable_adetailer", False):
+            return
+
+        if not self.is_ad_enabled(*args_):
+            return
+
+        p._ad_idx = getattr(p, "_ad_idx", -1) + 1
+        init_image = copy(pp.image)
+        arg_list = self.get_args(p, *args_)
+
+        if p.scripts is not None and self.need_call_postprocess(p):
+            dummy = Processed(p, [], p.seed, "")
+            with preseve_prompts(p):
+                p.scripts.postprocess(copy(p), dummy)
+
+        is_processed = False
+        with CNHijackRestore(), pause_total_tqdm(), cn_allow_script_control():
+            for n, args in enumerate(arg_list):
+                if args.ad_model == "None":
+                    continue
+                is_processed |= self._postprocess_image(p, pp, args, n=n)
+
+        if is_processed:
+            self.save_image(
+                p, init_image, condition="ad_save_images_before", suffix="-ad-before"
+            )
+
+        if p.scripts is not None and self.need_call_process(p):
+            with preseve_prompts(p):
+                p.scripts.process(copy(p))
+
+        try:
+            ia = p._ad_idx
+            lenp = len(p.all_prompts)
+            if ia % lenp == lenp - 1:
+                self.write_params_txt(p)
+        except Exception:
+            pass
+
+
+def on_after_component(component, **_kwargs):
+    global txt2img_submit_button, img2img_submit_button
+    if getattr(component, "elem_id", None) == "txt2img_generate":
+        txt2img_submit_button = component
+        return
+
+    if getattr(component, "elem_id", None) == "img2img_generate":
+        img2img_submit_button = component
+
+
+def on_ui_settings():
+    section = ("ADetailer", AFTER_DETAILER)
+    shared.opts.add_option(
+        "ad_max_models",
+        shared.OptionInfo(
+            default=2,
+            label="Max models",
+            component=gr.Slider,
+            component_args={"minimum": 1, "maximum": 10, "step": 1},
+            section=section,
+        ),
+    )
+
+    shared.opts.add_option(
+        "ad_save_previews",
+        shared.OptionInfo(False, "Save mask previews", section=section),
+    )
+
+    shared.opts.add_option(
+        "ad_save_images_before",
+        shared.OptionInfo(False, "Save images before ADetailer", section=section),
+    )
+
+    shared.opts.add_option(
+        "ad_only_seleted_scripts",
+        shared.OptionInfo(
+            True, "Apply only selected scripts to ADetailer", section=section
+        ),
+    )
+
+    textbox_args = {
+        "placeholder": "comma-separated list of script names",
+        "interactive": True,
+    }
+
+    shared.opts.add_option(
+        "ad_script_names",
+        shared.OptionInfo(
+            default=SCRIPT_DEFAULT,
+            label="Script names to apply to ADetailer (separated by comma)",
+            component=gr.Textbox,
+            component_args=textbox_args,
+            section=section,
+        ),
+    )
+
+    shared.opts.add_option(
+        "ad_bbox_sortby",
+        shared.OptionInfo(
+            default="None",
+            label="Sort bounding boxes by",
+            component=gr.Radio,
+            component_args={"choices": BBOX_SORTBY},
+            section=section,
+        ),
+    )
+
+
+# xyz_grid
+
+
+def make_axis_on_xyz_grid():
+    xyz_grid = None
+    for script in scripts.scripts_data:
+        if script.script_class.__module__ == "xyz_grid.py":
+            xyz_grid = script.module
+            break
+
+    if xyz_grid is None:
+        return
+
+    model_list = ["None", *model_mapping.keys()]
+
+    def set_value(p, x, xs, *, field: str):
+        if not hasattr(p, "adetailer_xyz"):
+            p.adetailer_xyz = {}
+        p.adetailer_xyz[field] = x
+
+    axis = [
+        xyz_grid.AxisOption(
+            "[ADetailer] ADetailer model 1st",
+            str,
+            partial(set_value, field="ad_model"),
+            choices=lambda: model_list,
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] ADetailer prompt 1st",
+            str,
+            partial(set_value, field="ad_prompt"),
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] ADetailer negative prompt 1st",
+            str,
+            partial(set_value, field="ad_negative_prompt"),
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] Mask erosion / dilation 1st",
+            int,
+            partial(set_value, field="ad_dilate_erode"),
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] Inpaint denoising strength 1st",
+            float,
+            partial(set_value, field="ad_denoising_strength"),
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] Inpaint only masked 1st",
+            str,
+            partial(set_value, field="ad_inpaint_only_masked"),
+            choices=lambda: ["True", "False"],
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] Inpaint only masked padding 1st",
+            int,
+            partial(set_value, field="ad_inpaint_only_masked_padding"),
+        ),
+        xyz_grid.AxisOption(
+            "[ADetailer] ControlNet model 1st",
+            str,
+            partial(set_value, field="ad_controlnet_model"),
+            choices=lambda: ["None", *get_cn_models()],
+        ),
+    ]
+
+    if not any(x.label.startswith("[ADetailer]") for x in xyz_grid.axis_options):
+        xyz_grid.axis_options.extend(axis)
+
+
+def on_before_ui():
+    try:
+        make_axis_on_xyz_grid()
+    except Exception:
+        error = traceback.format_exc()
+        print(
+            f"[-] ADetailer: xyz_grid error:\n{error}",
+            file=sys.stderr,
+        )
+
+
+script_callbacks.on_ui_settings(on_ui_settings)
+script_callbacks.on_after_component(on_after_component)
+script_callbacks.on_before_ui(on_before_ui)

extensions/adetailer/sd_webui/devices.py

@@ -0,0 +1,11 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+
+    class NansException(Exception):  # noqa: N818
+        pass
+
+else:
+    from modules.devices import NansException

extensions/adetailer/sd_webui/images.py

@@ -0,0 +1,62 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from PIL import Image, PngImagePlugin
+
+    from sd_webui.processing import StableDiffusionProcessing
+
+    def save_image(
+        image: Image.Image,
+        path: str,
+        basename: str,
+        seed: int | None = None,
+        prompt: str = "",
+        extension: str = "png",
+        info: str | PngImagePlugin.iTXt = "",
+        short_filename: bool = False,
+        no_prompt: bool = False,
+        grid: bool = False,
+        pnginfo_section_name: str = "parameters",
+        p: StableDiffusionProcessing | None = None,
+        existing_info: dict | None = None,
+        forced_filename: str | None = None,
+        suffix: str = "",
+        save_to_dirs: bool = False,
+    ) -> tuple[str, str | None]:
+        """Save an image.
+
+        Args:
+            image (`PIL.Image`):
+                The image to be saved.
+            path (`str`):
+                The directory to save the image. Note, the option `save_to_dirs` will make the image to be saved into a sub directory.
+            basename (`str`):
+                The base filename which will be applied to `filename pattern`.
+            seed, prompt, short_filename,
+            extension (`str`):
+                Image file extension, default is `png`.
+            pngsectionname (`str`):
+                Specify the name of the section which `info` will be saved in.
+            info (`str` or `PngImagePlugin.iTXt`):
+                PNG info chunks.
+            existing_info (`dict`):
+                Additional PNG info. `existing_info == {pngsectionname: info, ...}`
+            no_prompt:
+                TODO I don't know its meaning.
+            p (`StableDiffusionProcessing`)
+            forced_filename (`str`):
+                If specified, `basename` and filename pattern will be ignored.
+            save_to_dirs (bool):
+                If true, the image will be saved into a subdirectory of `path`.
+
+        Returns: (fullfn, txt_fullfn)
+            fullfn (`str`):
+                The full path of the saved imaged.
+            txt_fullfn (`str` or None):
+                If a text file is saved for this image, this will be its full path. Otherwise None.
+        """
+
+else:
+    from modules.images import save_image

extensions/adetailer/sd_webui/paths.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import os
+
+    models_path = os.path.join(os.path.dirname(__file__), "1")
+    script_path = os.path.join(os.path.dirname(__file__), "2")
+    data_path = os.path.join(os.path.dirname(__file__), "3")
+    extensions_dir = os.path.join(os.path.dirname(__file__), "4")
+    extensions_builtin_dir = os.path.join(os.path.dirname(__file__), "5")
+else:
+    from modules.paths import data_path, models_path, script_path

extensions/adetailer/sd_webui/processing.py

@@ -0,0 +1,176 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from dataclasses import dataclass, field
+    from typing import Any, Callable
+
+    import numpy as np
+    import torch
+    from PIL import Image
+
+    def _image():
+        return Image.new("L", (512, 512))
+
+    @dataclass
+    class StableDiffusionProcessing:
+        sd_model: torch.nn.Module = field(default_factory=lambda: torch.nn.Linear(1, 1))
+        outpath_samples: str = ""
+        outpath_grids: str = ""
+        prompt: str = ""
+        prompt_for_display: str = ""
+        negative_prompt: str = ""
+        styles: list[str] = field(default_factory=list)
+        seed: int = -1
+        subseed: int = -1
+        subseed_strength: float = 0.0
+        seed_resize_from_h: int = -1
+        seed_resize_from_w: int = -1
+        sampler_name: str | None = None
+        batch_size: int = 1
+        n_iter: int = 1
+        steps: int = 50
+        cfg_scale: float = 7.0
+        width: int = 512
+        height: int = 512
+        restore_faces: bool = False
+        tiling: bool = False
+        do_not_save_samples: bool = False
+        do_not_save_grid: bool = False
+        extra_generation_params: dict[str, Any] = field(default_factory=dict)
+        overlay_images: list[Image.Image] = field(default_factory=list)
+        eta: float = 0.0
+        do_not_reload_embeddings: bool = False
+        paste_to: tuple[int | float, ...] = (0, 0, 0, 0)
+        color_corrections: list[np.ndarray] = field(default_factory=list)
+        denoising_strength: float = 0.0
+        sampler_noise_scheduler_override: Callable | None = None
+        ddim_discretize: str = ""
+        s_min_uncond: float = 0.0
+        s_churn: float = 0.0
+        s_tmin: float = 0.0
+        s_tmax: float = 0.0
+        s_noise: float = 0.0
+        override_settings: dict[str, Any] = field(default_factory=dict)
+        override_settings_restore_afterwards: bool = False
+        is_using_inpainting_conditioning: bool = False
+        disable_extra_networks: bool = False
+        scripts: Any = None
+        script_args: list[Any] = field(default_factory=list)
+        all_prompts: list[str] = field(default_factory=list)
+        all_negative_prompts: list[str] = field(default_factory=list)
+        all_seeds: list[int] = field(default_factory=list)
+        all_subseeds: list[int] = field(default_factory=list)
+        iteration: int = 1
+        is_hr_pass: bool = False
+
+        def close(self) -> None:
+            pass
+
+    @dataclass
+    class StableDiffusionProcessingTxt2Img(StableDiffusionProcessing):
+        sampler: Callable | None = None
+        enable_hr: bool = False
+        denoising_strength: float = 0.75
+        hr_scale: float = 2.0
+        hr_upscaler: str = ""
+        hr_second_pass_steps: int = 0
+        hr_resize_x: int = 0
+        hr_resize_y: int = 0
+        hr_upscale_to_x: int = 0
+        hr_upscale_to_y: int = 0
+        width: int = 512
+        height: int = 512
+        truncate_x: int = 512
+        truncate_y: int = 512
+        applied_old_hires_behavior_to: tuple[int, int] = (512, 512)
+
+    @dataclass
+    class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
+        sampler: Callable | None = None
+        init_images: list[Image.Image] = field(default_factory=list)
+        resize_mode: int = 0
+        denoising_strength: float = 0.75
+        image_cfg_scale: float | None = None
+        init_latent: torch.Tensor | None = None
+        image_mask: Image.Image = field(default_factory=_image)
+        latent_mask: Image.Image = field(default_factory=_image)
+        mask_for_overlay: Image.Image = field(default_factory=_image)
+        mask_blur: int = 4
+        inpainting_fill: int = 0
+        inpaint_full_res: bool = True
+        inpaint_full_res_padding: int = 0
+        inpainting_mask_invert: int | bool = 0
+        initial_noise_multiplier: float = 1.0
+        mask: torch.Tensor | None = None
+        nmask: torch.Tensor | None = None
+        image_conditioning: torch.Tensor | None = None
+
+    @dataclass
+    class Processed:
+        images: list[Image.Image] = field(default_factory=list)
+        prompt: list[str] = field(default_factory=list)
+        negative_prompt: list[str] = field(default_factory=list)
+        seed: list[int] = field(default_factory=list)
+        subseed: list[int] = field(default_factory=list)
+        subseed_strength: float = 0.0
+        info: str = ""
+        comments: str = ""
+        width: int = 512
+        height: int = 512
+        sampler_name: str = ""
+        cfg_scale: float = 7.0
+        image_cfg_scale: float | None = None
+        steps: int = 50
+        batch_size: int = 1
+        restore_faces: bool = False
+        face_restoration_model: str | None = None
+        sd_model_hash: str = ""
+        seed_resize_from_w: int = -1
+        seed_resize_from_h: int = -1
+        denoising_strength: float = 0.0
+        extra_generation_params: dict[str, Any] = field(default_factory=dict)
+        index_of_first_image: int = 0
+        styles: list[str] = field(default_factory=list)
+        job_timestamp: str = ""
+        clip_skip: int = 1
+        eta: float = 0.0
+        ddim_discretize: str = ""
+        s_churn: float = 0.0
+        s_tmin: float = 0.0
+        s_tmax: float = 0.0
+        s_noise: float = 0.0
+        sampler_noise_scheduler_override: Callable | None = None
+        is_using_inpainting_conditioning: bool = False
+        all_prompts: list[str] = field(default_factory=list)
+        all_negative_prompts: list[str] = field(default_factory=list)
+        all_seeds: list[int] = field(default_factory=list)
+        all_subseeds: list[int] = field(default_factory=list)
+        infotexts: list[str] = field(default_factory=list)
+
+    def create_infotext(
+        p: StableDiffusionProcessingTxt2Img | StableDiffusionProcessingImg2Img,
+        all_prompts: list[str],
+        all_seeds: list[int],
+        all_subseeds: list[int],
+        comments: Any,
+        iteration: int = 0,
+        position_in_batch: int = 0,
+    ) -> str:
+        pass
+
+    def process_images(
+        p: StableDiffusionProcessingTxt2Img | StableDiffusionProcessingImg2Img,
+    ) -> Processed:
+        pass
+
+else:
+    from modules.processing import (
+        Processed,
+        StableDiffusionProcessing,
+        StableDiffusionProcessingImg2Img,
+        StableDiffusionProcessingTxt2Img,
+        create_infotext,
+        process_images,
+    )