glenux/code-preloader
1
0
Fork 0
Code Issues 19 Pull requests Packages Projects 1 Releases Wiki Activity

Update

Browse source
This commit is contained in:
Glenn Y. Rolland 2023-12-29 14:57:11 +01:00
parent 7da2465cad
commit e6ec6d9203
4 changed files with 130 additions and 46 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Makefile
View file

@ -2,4 +2,4 @@
all: build
build:
shards build
shards build --error-trace

85
README.md
View file

@ -1,30 +1,44 @@
# README
# Code-Preloader for ChatGPT
code-preloader is a tool that helps preloading all files for a given root directory
and concatenates them as a single file.
Welcome to Code-Preloader, a tool designed to enhance the efficiency of ChatGPT
by preloading all files within a specified root directory and concatenating
them into a single file. This tool is particularly useful for preparing smaller
codebases for processing by ChatGPT, as it allows for the inclusion of
customized prompts through header and footer files.
## Structure of the output file
## Features
```jinja2
{{ HEADER_PROMPT_FILE }}
* Preload and concatenate files from a given directory.
* Customizable header and footer prompts for ChatGPT.
* Simple command-line interface for easy operation.
{% for file in file_list %}
@@ File "{{ file.path }}"
{{ file.content }}
{% endfor %}
## Limitations
{{ FOOTER_PROMPT_FILE }}
**Note:** Code-Preloader is only suitable for smaller codebas. ChatGPT may not
perform as expected with large code repositories.
## Installation
To get started with Code-Preloader, ensure that you have Crystal language installed on your system. Follow these steps to install:
```bash
git clone https://github.com/your-repository/code-preloader.git
cd code-preloader
shards install
shards build
```
## Usage
Run Code-Preloader with the following command-line options:
```
Usage: code-preloader [options] ROOT_DIR
Options:
-c, --config=CONFIG_FILE Load parameters from CONFIG_FILE (ignore, output, etc.)
-i, --ignore=IGNORE_FILE Ignore file or directory path (not a pattern)
-o, --output=OUTPUT_FILE Write output to OUTPUT_FILE (default to stdout if option missing or OUTPUT_FILE is "-")
-c, --config=CONFIG_FILE Load parameters from CONFIG_FILE (not implemented yet)
-i, --ignore=IGNORE_FILE Ignore file or directory path
-o, --output=OUTPUT_FILE Write output to OUTPUT_FILE (default to stdout)
--header-prompt=HEADER_PROMPT_FILE Load header prompt from PROMPT_FILE
--footer-prompt=FOOTER_PROMPT_FILE Load footer prompt from PROMPT_FILE
--version Show version
@ -32,3 +46,46 @@ Options:
-h, --help Show this help
```
### Examples
To preload all files in the `src` directory and output to `result.txt`, while
ignoring the `git` the `bin` directory, and the result file itself:
```bash
./bin/code-preloader -o result.txt -i .git -i result.txt -i bin/ src
```
To preload all files in the `src` directory and output to clipboard, prepending
and appending prompts, while ignoring the `git` the `bin` directory, and the
result file itself:
```bash
./bin/code-preloader \
-o result.txt \
-i .git -i bin/ -i result.txt -i prompts \
--header-prompt prompts/header-context.txt \
--footer-prompt prompts/footer-write-readme.txt \
src \
| xclip -selection clipboard -i
```
## Contributing
Contributions are what make the open-source community such an amazing place to
learn, inspire, and create. Any contributions you make are **greatly
appreciated**.
## Troubleshooting and Support
If you encounter any issues or need support, please open an issue in the
project's GitHub issue tracker. We strive to be responsive and helpful.
## License
Distributed under the LGPL-3.0-or-later License. See `LICENSE` file for more
information.
## Acknowledgments
- Special thanks to all contributors and users of this project!

10
prompts/footer-improve-readme.txt Normal file
View file

@ -0,0 +1,10 @@
@@ REQUEST
We want to improve the README.md file as for an open-source project following the best practices.
Please focus on the code provided in the "@@ File..." sections and analyze the existing README.md.
In this analysis, please provide me with a list of things that you appreciate, a list of things to amplify, a list of things to adjust.

79
src/cli.cr
View file

@ -7,16 +7,15 @@ require "option_parser"
module CodePreloader
# The Cli class handles command-line interface operations for the CodePreloader.
class Cli
getter repo_path : String
getter repository_path : String?
getter ignore_list : Array(String) = [] of String
getter output_file_path : String
getter preamble_file_path : String?
getter output_file_path : String?
getter header_prompt_file_path : String? # Add type annotation
getter footer_prompt_file_path : String? # Assuming you'll also need this
# Initializes the Cli class with default values.
def initialize
@repo_path = ""
@repository_path = ""
@output_file_path = ""
end
@ -54,7 +53,7 @@ module CodePreloader
if remaining_args.size != 1
abort("Invalid number of arguments. Expected exactly one argument for ROOT_DIR.")
end
@repo_path = remaining_args[0]
@repository_path = remaining_args[0]
end
end
@ -63,21 +62,45 @@ module CodePreloader
# Executes the main functionality of the CLI application.
def exec
if preamble_file_path
STDERR.puts "Loading preamble from: #{preamble_file_path}"
header_prompt = ""
footer_prompt = ""
__header_prompt_file_path = @header_prompt_file_path
__footer_prompt_file_path = @footer_prompt_file_path
if !__header_prompt_file_path.nil?
STDERR.puts "Loading header prompt from: #{__header_prompt_file_path}"
header_prompt = File.read(__header_prompt_file_path)
end
if header_prompt_file_path
STDERR.puts "Loading header prompt from: #{header_prompt_file_path}"
if !__footer_prompt_file_path.nil?
STDERR.puts "Loading footer prompt from: #{__footer_prompt_file_path}"
footer_prompt = File.read(__footer_prompt_file_path)
end
if footer_prompt_file_path
STDERR.puts "Loading footer prompt from: #{footer_prompt_file_path}"
end
STDERR.puts "Processing repository: #{repository_path}"
STDERR.puts "Processing repository: #{repo_path}"
process_repository
STDERR.puts "Processing completed. Output written to: #{@output_file_path.empty? ? "stdout" : @output_file_path}"
__output_file_path = @output_file_path
__repository_path = @repository_path
abort("@output_file_path should be non-nil here") if __output_file_path.nil?
abort("@repository_path should be non-nil here") if __repository_path.nil?
invalid_output_file = true
output_file = STDOUT
unless __output_file_path.nil? || __output_file_path.try(&.empty?) || (__output_file_path != "-")
output_file = File.open(__output_file_path, "w")
invalid_output_file = false
end
output_file.puts header_prompt if header_prompt_file_path
process_repository(__repository_path, output_file)
output_file.puts footer_prompt if footer_prompt_file_path
output_file.close if !invalid_output_file
STDERR.puts "Processing completed. Output written to: #{invalid_output_file ? "stdout" : __output_file_path}"
rescue e : Exception
STDERR.puts "An error occurred during execution: #{e.message}"
@ -85,19 +108,9 @@ module CodePreloader
end
# Processes the specified repository and writes the output to a file.
def process_repository
local_output_file_path = @output_file_path
must_close = false
output_file = STDOUT
def process_repository(repository_path : String, output_file : IO::FileDescriptor)
if !local_output_file_path.empty? && (local_output_file_path != "-")
output_file = File.open(local_output_file_path, "w")
must_close = true
end
output_file.puts preamble_text if preamble_file_path
process_directory(repo_path, output_file)
output_file.close if must_close
process_directory(repository_path, output_file)
rescue e : IO::Error
STDERR.puts "Error processing repository: #{e.message}"
@ -115,7 +128,7 @@ module CodePreloader
)
next if !ignores.empty?
puts "File: #{child_path}"
STDERR.puts "File: #{child_path}"
child_path = File.join(path, child)
if File.directory?(child_path)
process_directory(child_path, output_file)
@ -126,7 +139,10 @@ module CodePreloader
end
private def process_file(file_path : String, output_file : IO::FileDescriptor)
relative_file_path = file_path.sub(/^#{Regex.escape(repo_path)}/, ".").lstrip
__repository_path = @repository_path
abort("@repository_path should be non-nil here") if __repository_path.nil?
relative_file_path = file_path.sub(/^#{Regex.escape(__repository_path)}/, ".").lstrip
output_file.puts "@@ File \"#{relative_file_path}\""
output_file.puts ""
output_file.puts(File.read(file_path))
@ -144,8 +160,9 @@ module CodePreloader
end
private def validate_arguments
abort("Missing repository path.") if repo_path.empty?
STDERR.puts("Output file path not specified (using STDOUT)") if output_file_path.empty?
abort("Missing repository path.") if @repository_path.nil? || @repository_path.try(&.empty?)
abort("Missing repository path.") if
STDERR.puts("Output file path not specified (using STDOUT)") if @output_file_path.nil? || @output_file_path.try(&.empty?)
end
# Reads and returns a list of paths to ignore from the given file.