Update
This commit is contained in:
parent
7da2465cad
commit
e6ec6d9203
4 changed files with 130 additions and 46 deletions
2
Makefile
2
Makefile
|
@ -2,4 +2,4 @@
|
||||||
all: build
|
all: build
|
||||||
|
|
||||||
build:
|
build:
|
||||||
shards build
|
shards build --error-trace
|
||||||
|
|
85
README.md
85
README.md
|
@ -1,30 +1,44 @@
|
||||||
# README
|
# Code-Preloader for ChatGPT
|
||||||
|
|
||||||
code-preloader is a tool that helps preloading all files for a given root directory
|
Welcome to Code-Preloader, a tool designed to enhance the efficiency of ChatGPT
|
||||||
and concatenates them as a single file.
|
by preloading all files within a specified root directory and concatenating
|
||||||
|
them into a single file. This tool is particularly useful for preparing smaller
|
||||||
|
codebases for processing by ChatGPT, as it allows for the inclusion of
|
||||||
|
customized prompts through header and footer files.
|
||||||
|
|
||||||
## Structure of the output file
|
## Features
|
||||||
|
|
||||||
```jinja2
|
* Preload and concatenate files from a given directory.
|
||||||
{{ HEADER_PROMPT_FILE }}
|
* Customizable header and footer prompts for ChatGPT.
|
||||||
|
* Simple command-line interface for easy operation.
|
||||||
|
|
||||||
{% for file in file_list %}
|
## Limitations
|
||||||
@@ File "{{ file.path }}"
|
|
||||||
{{ file.content }}
|
|
||||||
{% endfor %}
|
|
||||||
|
|
||||||
{{ FOOTER_PROMPT_FILE }}
|
**Note:** Code-Preloader is only suitable for smaller codebas. ChatGPT may not
|
||||||
|
perform as expected with large code repositories.
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
To get started with Code-Preloader, ensure that you have Crystal language installed on your system. Follow these steps to install:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/your-repository/code-preloader.git
|
||||||
|
cd code-preloader
|
||||||
|
shards install
|
||||||
|
shards build
|
||||||
```
|
```
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
|
Run Code-Preloader with the following command-line options:
|
||||||
|
|
||||||
```
|
```
|
||||||
Usage: code-preloader [options] ROOT_DIR
|
Usage: code-preloader [options] ROOT_DIR
|
||||||
|
|
||||||
Options:
|
Options:
|
||||||
-c, --config=CONFIG_FILE Load parameters from CONFIG_FILE (ignore, output, etc.)
|
-c, --config=CONFIG_FILE Load parameters from CONFIG_FILE (not implemented yet)
|
||||||
-i, --ignore=IGNORE_FILE Ignore file or directory path (not a pattern)
|
-i, --ignore=IGNORE_FILE Ignore file or directory path
|
||||||
-o, --output=OUTPUT_FILE Write output to OUTPUT_FILE (default to stdout if option missing or OUTPUT_FILE is "-")
|
-o, --output=OUTPUT_FILE Write output to OUTPUT_FILE (default to stdout)
|
||||||
--header-prompt=HEADER_PROMPT_FILE Load header prompt from PROMPT_FILE
|
--header-prompt=HEADER_PROMPT_FILE Load header prompt from PROMPT_FILE
|
||||||
--footer-prompt=FOOTER_PROMPT_FILE Load footer prompt from PROMPT_FILE
|
--footer-prompt=FOOTER_PROMPT_FILE Load footer prompt from PROMPT_FILE
|
||||||
--version Show version
|
--version Show version
|
||||||
|
@ -32,3 +46,46 @@ Options:
|
||||||
-h, --help Show this help
|
-h, --help Show this help
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
To preload all files in the `src` directory and output to `result.txt`, while
|
||||||
|
ignoring the `git` the `bin` directory, and the result file itself:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/code-preloader -o result.txt -i .git -i result.txt -i bin/ src
|
||||||
|
```
|
||||||
|
|
||||||
|
To preload all files in the `src` directory and output to clipboard, prepending
|
||||||
|
and appending prompts, while ignoring the `git` the `bin` directory, and the
|
||||||
|
result file itself:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/code-preloader \
|
||||||
|
-o result.txt \
|
||||||
|
-i .git -i bin/ -i result.txt -i prompts \
|
||||||
|
--header-prompt prompts/header-context.txt \
|
||||||
|
--footer-prompt prompts/footer-write-readme.txt \
|
||||||
|
src \
|
||||||
|
| xclip -selection clipboard -i
|
||||||
|
```
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
Contributions are what make the open-source community such an amazing place to
|
||||||
|
learn, inspire, and create. Any contributions you make are **greatly
|
||||||
|
appreciated**.
|
||||||
|
|
||||||
|
## Troubleshooting and Support
|
||||||
|
|
||||||
|
If you encounter any issues or need support, please open an issue in the
|
||||||
|
project's GitHub issue tracker. We strive to be responsive and helpful.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Distributed under the LGPL-3.0-or-later License. See `LICENSE` file for more
|
||||||
|
information.
|
||||||
|
|
||||||
|
## Acknowledgments
|
||||||
|
|
||||||
|
- Special thanks to all contributors and users of this project!
|
||||||
|
|
||||||
|
|
10
prompts/footer-improve-readme.txt
Normal file
10
prompts/footer-improve-readme.txt
Normal file
|
@ -0,0 +1,10 @@
|
||||||
|
@@ REQUEST
|
||||||
|
|
||||||
|
|
||||||
|
We want to improve the README.md file as for an open-source project following the best practices.
|
||||||
|
|
||||||
|
Please focus on the code provided in the "@@ File..." sections and analyze the existing README.md.
|
||||||
|
|
||||||
|
In this analysis, please provide me with a list of things that you appreciate, a list of things to amplify, a list of things to adjust.
|
||||||
|
|
||||||
|
|
77
src/cli.cr
77
src/cli.cr
|
@ -7,16 +7,15 @@ require "option_parser"
|
||||||
module CodePreloader
|
module CodePreloader
|
||||||
# The Cli class handles command-line interface operations for the CodePreloader.
|
# The Cli class handles command-line interface operations for the CodePreloader.
|
||||||
class Cli
|
class Cli
|
||||||
getter repo_path : String
|
getter repository_path : String?
|
||||||
getter ignore_list : Array(String) = [] of String
|
getter ignore_list : Array(String) = [] of String
|
||||||
getter output_file_path : String
|
getter output_file_path : String?
|
||||||
getter preamble_file_path : String?
|
|
||||||
getter header_prompt_file_path : String? # Add type annotation
|
getter header_prompt_file_path : String? # Add type annotation
|
||||||
getter footer_prompt_file_path : String? # Assuming you'll also need this
|
getter footer_prompt_file_path : String? # Assuming you'll also need this
|
||||||
|
|
||||||
# Initializes the Cli class with default values.
|
# Initializes the Cli class with default values.
|
||||||
def initialize
|
def initialize
|
||||||
@repo_path = ""
|
@repository_path = ""
|
||||||
@output_file_path = ""
|
@output_file_path = ""
|
||||||
end
|
end
|
||||||
|
|
||||||
|
@ -54,7 +53,7 @@ module CodePreloader
|
||||||
if remaining_args.size != 1
|
if remaining_args.size != 1
|
||||||
abort("Invalid number of arguments. Expected exactly one argument for ROOT_DIR.")
|
abort("Invalid number of arguments. Expected exactly one argument for ROOT_DIR.")
|
||||||
end
|
end
|
||||||
@repo_path = remaining_args[0]
|
@repository_path = remaining_args[0]
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
@ -63,21 +62,45 @@ module CodePreloader
|
||||||
|
|
||||||
# Executes the main functionality of the CLI application.
|
# Executes the main functionality of the CLI application.
|
||||||
def exec
|
def exec
|
||||||
if preamble_file_path
|
header_prompt = ""
|
||||||
STDERR.puts "Loading preamble from: #{preamble_file_path}"
|
footer_prompt = ""
|
||||||
|
__header_prompt_file_path = @header_prompt_file_path
|
||||||
|
__footer_prompt_file_path = @footer_prompt_file_path
|
||||||
|
|
||||||
|
if !__header_prompt_file_path.nil?
|
||||||
|
STDERR.puts "Loading header prompt from: #{__header_prompt_file_path}"
|
||||||
|
header_prompt = File.read(__header_prompt_file_path)
|
||||||
end
|
end
|
||||||
|
|
||||||
if header_prompt_file_path
|
if !__footer_prompt_file_path.nil?
|
||||||
STDERR.puts "Loading header prompt from: #{header_prompt_file_path}"
|
STDERR.puts "Loading footer prompt from: #{__footer_prompt_file_path}"
|
||||||
|
footer_prompt = File.read(__footer_prompt_file_path)
|
||||||
end
|
end
|
||||||
|
|
||||||
if footer_prompt_file_path
|
STDERR.puts "Processing repository: #{repository_path}"
|
||||||
STDERR.puts "Loading footer prompt from: #{footer_prompt_file_path}"
|
|
||||||
|
__output_file_path = @output_file_path
|
||||||
|
__repository_path = @repository_path
|
||||||
|
|
||||||
|
abort("@output_file_path should be non-nil here") if __output_file_path.nil?
|
||||||
|
abort("@repository_path should be non-nil here") if __repository_path.nil?
|
||||||
|
|
||||||
|
invalid_output_file = true
|
||||||
|
output_file = STDOUT
|
||||||
|
|
||||||
|
unless __output_file_path.nil? || __output_file_path.try(&.empty?) || (__output_file_path != "-")
|
||||||
|
output_file = File.open(__output_file_path, "w")
|
||||||
|
invalid_output_file = false
|
||||||
end
|
end
|
||||||
|
|
||||||
STDERR.puts "Processing repository: #{repo_path}"
|
output_file.puts header_prompt if header_prompt_file_path
|
||||||
process_repository
|
|
||||||
STDERR.puts "Processing completed. Output written to: #{@output_file_path.empty? ? "stdout" : @output_file_path}"
|
process_repository(__repository_path, output_file)
|
||||||
|
|
||||||
|
output_file.puts footer_prompt if footer_prompt_file_path
|
||||||
|
|
||||||
|
output_file.close if !invalid_output_file
|
||||||
|
STDERR.puts "Processing completed. Output written to: #{invalid_output_file ? "stdout" : __output_file_path}"
|
||||||
|
|
||||||
rescue e : Exception
|
rescue e : Exception
|
||||||
STDERR.puts "An error occurred during execution: #{e.message}"
|
STDERR.puts "An error occurred during execution: #{e.message}"
|
||||||
|
@ -85,19 +108,9 @@ module CodePreloader
|
||||||
end
|
end
|
||||||
|
|
||||||
# Processes the specified repository and writes the output to a file.
|
# Processes the specified repository and writes the output to a file.
|
||||||
def process_repository
|
def process_repository(repository_path : String, output_file : IO::FileDescriptor)
|
||||||
local_output_file_path = @output_file_path
|
|
||||||
must_close = false
|
|
||||||
output_file = STDOUT
|
|
||||||
|
|
||||||
if !local_output_file_path.empty? && (local_output_file_path != "-")
|
process_directory(repository_path, output_file)
|
||||||
output_file = File.open(local_output_file_path, "w")
|
|
||||||
must_close = true
|
|
||||||
end
|
|
||||||
|
|
||||||
output_file.puts preamble_text if preamble_file_path
|
|
||||||
process_directory(repo_path, output_file)
|
|
||||||
output_file.close if must_close
|
|
||||||
|
|
||||||
rescue e : IO::Error
|
rescue e : IO::Error
|
||||||
STDERR.puts "Error processing repository: #{e.message}"
|
STDERR.puts "Error processing repository: #{e.message}"
|
||||||
|
@ -115,7 +128,7 @@ module CodePreloader
|
||||||
)
|
)
|
||||||
next if !ignores.empty?
|
next if !ignores.empty?
|
||||||
|
|
||||||
puts "File: #{child_path}"
|
STDERR.puts "File: #{child_path}"
|
||||||
child_path = File.join(path, child)
|
child_path = File.join(path, child)
|
||||||
if File.directory?(child_path)
|
if File.directory?(child_path)
|
||||||
process_directory(child_path, output_file)
|
process_directory(child_path, output_file)
|
||||||
|
@ -126,7 +139,10 @@ module CodePreloader
|
||||||
end
|
end
|
||||||
|
|
||||||
private def process_file(file_path : String, output_file : IO::FileDescriptor)
|
private def process_file(file_path : String, output_file : IO::FileDescriptor)
|
||||||
relative_file_path = file_path.sub(/^#{Regex.escape(repo_path)}/, ".").lstrip
|
__repository_path = @repository_path
|
||||||
|
abort("@repository_path should be non-nil here") if __repository_path.nil?
|
||||||
|
|
||||||
|
relative_file_path = file_path.sub(/^#{Regex.escape(__repository_path)}/, ".").lstrip
|
||||||
output_file.puts "@@ File \"#{relative_file_path}\""
|
output_file.puts "@@ File \"#{relative_file_path}\""
|
||||||
output_file.puts ""
|
output_file.puts ""
|
||||||
output_file.puts(File.read(file_path))
|
output_file.puts(File.read(file_path))
|
||||||
|
@ -144,8 +160,9 @@ module CodePreloader
|
||||||
end
|
end
|
||||||
|
|
||||||
private def validate_arguments
|
private def validate_arguments
|
||||||
abort("Missing repository path.") if repo_path.empty?
|
abort("Missing repository path.") if @repository_path.nil? || @repository_path.try(&.empty?)
|
||||||
STDERR.puts("Output file path not specified (using STDOUT)") if output_file_path.empty?
|
abort("Missing repository path.") if
|
||||||
|
STDERR.puts("Output file path not specified (using STDOUT)") if @output_file_path.nil? || @output_file_path.try(&.empty?)
|
||||||
end
|
end
|
||||||
|
|
||||||
# Reads and returns a list of paths to ignore from the given file.
|
# Reads and returns a list of paths to ignore from the given file.
|
||||||
|
|
Loading…
Reference in a new issue