diff --git a/LICENSE.txt b/LICENSE.txt
deleted file mode 100644
index f42af47..0000000
--- a/LICENSE.txt
+++ /dev/null
@@ -1,22 +0,0 @@
-MIT License
-
-Copyright (c) 2012-2024 Scott Chacon and others
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
diff --git a/NOTES.md b/NOTES.md
new file mode 100644
index 0000000..1c48ac4
--- /dev/null
+++ b/NOTES.md
@@ -0,0 +1,33 @@
+## Notes on some refactoring.
+
+- The goal is to bring more encapsulation of the models management and simplified configuration management to bring increased flexibility, transparency on the overall flow, and simplicity in adding new model.
+- We need to differentiate:
+ - Vendors: the producer of models (like OpenAI, Anthropric, Ollama, ..etc) and their associated APIs
+ - Models: the LLM models these vendors are making public
+- Each vendor and operations allowed by the vendor needs to be encapsulated. This includes:
+ - The questions needed to setup the model (like the API key, or the URL)
+ - The listing of all models supported by the vendor
+ - The actions performed with a given model
+
+- The configuration flow works like this for an **initial** call:
+ - The available vendors are called one by one, each of them being responsible for the data they collect. They return a set of environment variables under the form of a list of strings, or an empty list if the user does not want to setup this vendor. As we do not want each vendor to know which way the data they need will be collected (e.g., read from the command line, or a GUI), they will be asked for a list of questions, the configuration will inquire the user, and send back the questions with tthe collected answers to the Vendor. The Vendor is then either instantiating an instance (Vendor configured) and returning it, or returning `nil` if the Vendor should not be set up.
+ - the `.env` file is created, using the information returned by the vendors
+ - A list of patterns is downloaded from the main site
+
+- When the system is configured, the configuration flows:
+ - Read the `.env` file using the godotenv library
+ - It configures a structure that contains the various vendors selected as well as the preferred model. This structure will be completed with some of the command line values (i.e, context, session, etc..)
+
+- To get the list of all supported models:
+ - Each configured model (part of the configuration structure) is asked, using a goroutine, to return the list of model
+
+- Order when building message: session + context + pattern + user input (role "user)
+
+
+## TODO:
+- Check if we need to read the system.md for every patterns when runnign the ListAllPatterns
+- Context management seems more complex than the one in the original fabric. Probably needs some work (at least to make it clear how it works)
+- models on command line: give as well vendor (like `--model openai/gpt-4o`). If the vendor is not given, get it by retrieving all possible models and searching from that.
+- if user gives the ollama url on command line, we need to update/init an ollama vendor.
+- The db should host only things related to access and storage in ~/.config/fabric
+- The interaction part of the Setup function should be in the cli (and perhaps all the Setup)
diff --git a/README.md b/README.md
index 0558b15..1fb9844 100644
--- a/README.md
+++ b/README.md
@@ -14,8 +14,8 @@
fabric is an open-source framework for augmenting humans using AI.
-[Introduction Video](#introduction-video-by-network-chuck) •
-[What and Why](#what-and-why) •
+[Introduction Video](#introduction-video) •
+[What and Why](#whatandwhy) •
[Philosophy](#philosophy) •
[Quickstart](#quickstart) •
[Structure](#structure) •
@@ -29,7 +29,7 @@
## Navigation
-- [Introduction Videos](#introduction-video-by-network-chuck)
+- [Introduction Videos](#introduction-videos)
- [What and Why](#what-and-why)
- [Philosophy](#philosophy)
- [Breaking problems into components](#breaking-problems-into-components)
@@ -40,7 +40,6 @@
- [Using the fabric client](#using-the-fabric-client)
- [Just use the Patterns](#just-use-the-patterns)
- [Create your own Fabric Mill](#create-your-own-fabric-mill)
-- [Updating](#updating)
- [Structure](#structure)
- [Components](#components)
- [CLI-native](#cli-native)
@@ -54,14 +53,32 @@
> [!NOTE]
-> May 23, 2024 — We will be switching Fabric to Go in a few weeks to avoid all the installation issues with Python. The Go version will be dead-simple to install and will be even faster. Plus easier to update. We already have it working thanks to the heroic efforts of @xssdoctor, and we're just working on testing now! Stay tuned for more info on the release date!
+> We are adding functionality to the project so often that you should update often as well. That means: `go install github.com/danielmiessler/fabric` in the main directory!
-## Introduction video by Network Chuck!
+## Introduction videos
-This is a **brilliant** video by Network Chuck that goes over why he's started using Fabric for all things AI. He talks about the spirit of the project, how to install it, and how he uses it, and he just generally articulates the spirit of what we're doing here SO WELL. Thanks to Chuck for this!
+> [!NOTE]
+> We have recently migrated to go. If you are migrating for the first time. please run
-
-
+```
+bash
+
+pipx uninstall fabric
+go install github.com/danielmiessler/fabric
+fabric --setup // THIS IS IMPORTANT AS THERE ARE ELEMENTS OF THE CONFIG THAT HAVE CHANGED
+```
+
+
+
+
+
+
## What and why
@@ -94,15 +111,15 @@ One of fabric's primary features is helping people collect and inte
Fabric has Patterns for all sorts of life and work activities, including:
-- Extracting the most interesting parts of YouTube videos and podcasts.
-- Writing an essay in your own voice with just an idea as an input.
-- Summarizing opaque academic papers.
-- Creating perfectly matched AI art prompts for a piece of writing.
-- Rating the quality of content to see if you want to read/watch the whole thing.
-- Getting summaries of long, boring content.
-- Explaining code to you.
-- Turning bad documentation into usable documentation.
-- Creating social media posts from any content input.
+- Extracting the most interesting parts of YouTube videos and podcasts
+- Writing an essay in your own voice with just an idea as an input
+- Summarizing opaque academic papers
+- Creating perfectly matched AI art prompts for a piece of writing
+- Rating the quality of content to see if you want to read/watch the whole thing
+- Getting summaries of long, boring content
+- Explaining code to you
+- Turning bad documentation into usable documentation
+- Creating social media posts from any content input
- And a million more…
### Our approach to prompting
@@ -111,7 +128,7 @@ Fabric _Patterns_ are different than most prompts you'll see.
- **First, we use `Markdown` to help ensure maximum readability and editability**. This not only helps the creator make a good one, but also anyone who wants to deeply understand what it does. _Importantly, this also includes the AI you're sending it to!_
-Here's an example of a Fabric Pattern
+Here's an example of a Fabric Pattern.
```bash
https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/system.md
@@ -125,144 +142,87 @@ https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/syste
## Quickstart
-The most feature-rich way to use Fabric is to use the `fabric` client, which can be found under `/client` directory in this repository.
+The most feature-rich way to use Fabric is to use the `fabric` client, which can be found under `/client` directory in this repository.
-### Required Python Version
-Ensure you have at least python3.10 installed on your operating system. Otherwise, when you attempt to run the pip install commands, the project will fail to build due to certain dependencies.
+### Installation
-### Setting up the fabric commands
+To install Go, visit
-Follow these steps to get all fabric-related apps installed and configured.
-
-1. Navigate to where you want the Fabric project to live on your system in a semi-permanent place on your computer.
+https://go.dev/doc/install
```bash
-# Find a home for Fabric
-cd /where/you/keep/code
-```
+# Install fabric
-2. Clone the project to your computer.
-
-```bash
-# Clone Fabric to your computer
-git clone https://github.com/danielmiessler/fabric.git
-```
-
-3. Enter Fabric's main directory.
-
-```bash
-# Enter the project folder (where you cloned it)
-cd fabric
-```
-
-4. Install pipx:
-
-macOS:
-
-```bash
-brew install pipx
-```
-
-Linux:
-
-```bash
-sudo apt install pipx
-```
-
-Windows:
-
-Use WSL and follow the Linux instructions.
-
-5. Install fabric:
-
-```bash
-pipx install .
-```
-
-6. Run setup:
-
-```bash
-fabric --setup
-```
-
-7. Restart your shell to reload everything.
-
-8. Now you are up and running! You can test by running the help.
-
-```bash
-# Making sure the paths are set up correctly
-fabric --help
+go install github.com/danielmiessler/fabric
```
> [!NOTE]
-> If you're using the `server` functions, `fabric-api` and `fabric-webui` need to be run in distinct terminal windows.
-
-## Updating
-
-To update Fabric, run the following commands.
-
-```bash
-# From the fabric directory
-pipx install . --force
-fabric --update
-```
-
-Then restart your shell.
+> the gui, the server and all of the helpers have have been migrated to different repositiories. please visit...
### Using the `fabric` client
-If you want to use it with OpenAI API-compatible inference servers, such as [FastChat](https://github.com/lm-sys/FastChat), [Helmholtz Blablador](http://helmholtz-blablador.fz-juelich.de), [LM Studio](https://lmstudio.ai) and others, simply export the following environment variables:
-
-- `export OPENAI_BASE_URL=https://YOUR-SERVER:8000/v1/`
-- `export DEFAULT_MODEL="YOUR_MODEL"`
-
-And if your server needs authentication tokens, as Blablador does, you export the token the same way you would with OpenAI:
-
-- `export OPENAI_API_KEY="YOUR TOKEN"`
-
-Once you have it all set up, here's how to use it:
+Once you have it all set up, here's how to use it.
1. Check out the options
`fabric -h`
```bash
-usage: fabric -h
-usage: fabric [-h] [--text TEXT] [--copy] [--agents] [--output [OUTPUT]] [--session [SESSION]] [--gui] [--stream] [--list] [--temp TEMP] [--top_p TOP_P] [--frequency_penalty FREQUENCY_PENALTY]
- [--presence_penalty PRESENCE_PENALTY] [--update] [--pattern PATTERN] [--setup] [--changeDefaultModel CHANGEDEFAULTMODEL] [--model MODEL] [--listmodels]
- [--remoteOllamaServer REMOTEOLLAMASERVER] [--context]
+usage: fabric-go -h
+Usage:
+ fabric-go [OPTIONS]
-An open-source framework for augmenting humans using AI.
+Application Options:
+ -p, --pattern= Choose a pattern
+ -C, --context= Choose a context
+ --session= Choose a session
+ -S, --setup Run setup
+ -t, --temperature= Set temperature (default: 0.7)
+ -T, --topp= Set top P (default: 0.9)
+ -s, --stream Stream
+ -P, --presencepenalty= Set presence penalty (default: 0.0)
+ -F, --frequencypenalty= Set frequency penalty (default: 0.0)
+ -l, --listpatterns List all patterns
+ -L, --listmodels List all available models
+ -x, --listcontexts List all contexts
+ -X, --listsessions List all sessions
+ -U, --updatepatterns Update patterns
+ -A, --addcontext Add a context
+ -c, --copy Copy to clipboard
+ -m, --model= Choose model
+ -u, --url= Choose ollama url (default: http://127.0.0.1:11434)
+ -o, --output= Output to file
+ -n, --latest= Number of latest patterns to list (default: 0)
-options:
- -h, --help show this help message and exit
- --text TEXT, -t TEXT Text to extract summary from
- --copy, -C Copy the response to the clipboard
- --agents, -a Use praisonAI to create an AI agent and then use it. ex: 'write me a movie script'
- --output [OUTPUT], -o [OUTPUT]
- Save the response to a file
- --session [SESSION], -S [SESSION]
- Continue your previous conversation. Default is your previous conversation
- --gui Use the GUI (Node and npm need to be installed)
- --stream, -s Use this option if you want to see the results in realtime. NOTE: You will not be able to pipe the output into another command.
- --list, -l List available patterns
- --temp TEMP sets the temperature for the model. Default is 0
- --top_p TOP_P set the top_p for the model. Default is 1
- --frequency_penalty FREQUENCY_PENALTY
- sets the frequency penalty for the model. Default is 0.1
- --presence_penalty PRESENCE_PENALTY
- sets the presence penalty for the model. Default is 0.1
- --update, -u Update patterns.
- --pattern PATTERN, -p PATTERN
- The pattern (prompt) to use
- --setup Set up your fabric instance
- --changeDefaultModel CHANGEDEFAULTMODEL
- Change the default model. For a list of available models, use the --listmodels flag.
- --model MODEL, -m MODEL
- Select the model to use
- --listmodels List all available models
- --remoteOllamaServer REMOTEOLLAMASERVER
- The URL of the remote ollamaserver to use. ONLY USE THIS if you are using a local ollama server in a non-default location or port
- --context, -c Use Context file (context.md) to add context to your pattern
+Help Options:
+ -h, --help Show this help message
+
+Usage:
+ fabric-go [OPTIONS]
+
+Application Options:
+ -p, --pattern= Choose a pattern
+ -C, --context= Choose a context
+ --session= Choose a session
+ -S, --setup Run setup
+ -t, --temperature= Set temperature (default: 0.7)
+ -T, --topp= Set top P (default: 0.9)
+ -s, --stream Stream
+ -P, --presencepenalty= Set presence penalty (default: 0.0)
+ -F, --frequencypenalty= Set frequency penalty (default: 0.0)
+ -l, --listpatterns List all patterns
+ -L, --listmodels List all available models
+ -x, --listcontexts List all contexts
+ -X, --listsessions List all sessions
+ -U, --updatepatterns Update patterns
+ -A, --addcontext Add a context
+ -c, --copy Copy to clipboard
+ -m, --model= Choose model
+ -u, --url= Choose ollama url (default: http://127.0.0.1:11434)
+ -o, --output= Output to file
+ -n, --latest= Number of latest patterns to list (default: 0)
+
+Help Options:
+ -h, --help Show this help message
```
#### Example commands
@@ -287,14 +247,25 @@ pbpaste | fabric --stream --pattern analyze_claims
yt --transcript https://youtube.com/watch?v=uXs-zPc63kM | fabric --stream --pattern extract_wisdom
```
-4. **new** All of the patterns have been added as aliases to your bash (or zsh) config file
+4. create patterns- you must create a .md file with the pattern and save it to ~/.config/fabric/pattterns/[yourpatternname].
-```bash
-pbpaste | analyze_claims --stream
+5. create contexts- you must create a .txt file with the context and then run the following command
+
+```
+bash
+
+fabric --addcontext
```
-> [!NOTE]
-> More examples coming in the next few days, including a demo video!
+6. Sessions- sessions are persistant conversations. You can create a session by running the following command
+
+```
+bash
+
+echo 'my name is ben' | fabric --session ben
+```
+
+7. List
### Just use the Patterns
@@ -310,111 +281,6 @@ You can use any of the Patterns you see there in any AI application that you hav
The wisdom of crowds for the win.
-### Create your own Fabric Mill
-
-
-
-
-
-But we go beyond just providing Patterns. We provide code for you to build your very own Fabric server and personal AI infrastructure!
-
-## Structure
-
-Fabric is themed off of, well… _fabric_—as in…woven materials. So, think blankets, quilts, patterns, etc. Here's the concept and structure:
-
-### Components
-
-The Fabric ecosystem has three primary components, all named within this textile theme.
-
-- The **Mill** is the (optional) server that makes **Patterns** available.
-- **Patterns** are the actual granular AI use cases (prompts).
-- **Stitches** are chained together _Patterns_ that create advanced functionality (see below).
-- **Looms** are the client-side apps that call a specific **Pattern** hosted by a **Mill**.
-
-### CLI-native
-
-One of the coolest parts of the project is that it's **command-line native**!
-
-Each Pattern you see in the `/patterns` directory can be used in any AI application you use, but you can also set up your own server using the `/server` code and then call APIs directly!
-
-Once you're set-up, you can do things like:
-
-```bash
-# Take any idea from `stdin` and send it to the `/write_essay` API!
-echo "An idea that coding is like speaking with rules." | write_essay
-```
-
-### Directly calling Patterns
-
-One key feature of `fabric` and its Markdown-based format is the ability to _directly reference_ (and edit) individual [Patterns](#components) directly—on their own—without any surrounding code.
-
-As an example, here's how to call _the direct location_ of the `extract_wisdom` pattern.
-
-```bash
-https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/system.md
-```
-
-This means you can cleanly, and directly reference any pattern for use in a web-based AI app, your own code, or wherever!
-
-Even better, you can also have your [Mill](#components) functionality directly call _system_ and _user_ prompts from `fabric`, meaning you can have your personal AI ecosystem automatically kept up to date with the latest version of your favorite [Patterns](#components).
-
-Here's what that looks like in code:
-
-```bash
-https://github.com/danielmiessler/fabric/blob/main/server/fabric_api_server.py
-```
-
-```python
-# /extwis
-@app.route("/extwis", methods=["POST"])
-@auth_required # Require authentication
-def extwis():
- data = request.get_json()
-
- # Warn if there's no input
- if "input" not in data:
- return jsonify({"error": "Missing input parameter"}), 400
-
- # Get data from client
- input_data = data["input"]
-
- # Set the system and user URLs
- system_url = "https://raw.githubusercontent.com/danielmiessler/fabric/main/patterns/extract_wisdom/system.md"
- user_url = "https://raw.githubusercontent.com/danielmiessler/fabric/main/patterns/extract_wisdom/user.md"
-
- # Fetch the prompt content
- system_content = fetch_content_from_url(system_url)
- user_file_content = fetch_content_from_url(user_url)
-
- # Build the API call
- system_message = {"role": "system", "content": system_content}
- user_message = {"role": "user", "content": user_file_content + "\n" + input_data}
- messages = [system_message, user_message]
- try:
- response = openai.chat.completions.create(
- model="gpt-4-1106-preview",
- messages=messages,
- temperature=0.0,
- top_p=1,
- frequency_penalty=0.1,
- presence_penalty=0.1,
- )
- assistant_message = response.choices[0].message.content
- return jsonify({"response": assistant_message})
- except Exception as e:
- return jsonify({"error": str(e)}), 500
-```
-
-## Examples
-
-Here's an abridged output example from the `extract_wisdom` pattern (limited to only 10 items per section).
-
-```bash
-# Paste in the transcript of a YouTube video of Riva Tez on David Perrel's podcast
-pbpaste | extract_wisdom
-```
-
-```markdown
## SUMMARY:
The content features a conversation between two individuals discussing various topics, including the decline of Western culture, the importance of beauty and subtlety in life, the impact of technology and AI, the resonance of Rilke's poetry, the value of deep reading and revisiting texts, the captivating nature of Ayn Rand's writing, the role of philosophy in understanding the world, and the influence of drugs on society. They also touch upon creativity, attention spans, and the importance of introspection.
@@ -436,7 +302,7 @@ The content features a conversation between two individuals discussing various t
1. "You can't necessarily think yourself into the answers. You have to create space for the answers to come to you."
2. "The West is dying and we are killing her."
-3. "The American Dream has been replaced by mass-packaged mediocrity porn, encouraging us to revel like happy pigs in our own meekness."
+3. "The American Dream has been replaced by mass packaged mediocrity porn, encouraging us to revel like happy pigs in our own meekness."
4. "There's just not that many people who have the courage to reach beyond consensus and go explore new ideas."
5. "I'll start watching Netflix when I've read the whole of human history."
6. "Rilke saw beauty in everything... He sees it's in one little thing, a representation of all things that are beautiful."
@@ -483,139 +349,27 @@ The content features a conversation between two individuals discussing various t
8. Robert Pirsig's writings
9. Bertrand Russell's definition of philosophy
10. Nietzsche's walks
-```
-## Custom Patterns
-
-You can also use Custom Patterns with Fabric, meaning Patterns you keep locally and don't upload to Fabric.
-
-One possible place to store them is `~/.config/custom-fabric-patterns`.
-
-Then when you want to use them, simply copy them into `~/.config/fabric/patterns`.
-
-```bash
-cp -a ~/.config/custom-fabric-patterns/* ~/.config/fabric/patterns/
-```
-
-Now you can run them with:
-
-```bash
-pbpaste | fabric -p your_custom_pattern
-```
+````
## Agents
-NEW FEATURE! We have incorporated [PraisonAI](https://github.com/MervinPraison/PraisonAI) into Fabric. This feature creates AI agents and then uses them to perform a task.
+NEW FEATURE! We have incorporated PraisonAI with fabric. For more information about this amazing project please visit https://github.com/MervinPraison/PraisonAI. This feature CREATES AI agents and then uses them to perform a task
```bash
-echo "Search for recent articles about the future of AI and write me a 500-word essay on the findings" | fabric --agents
-```
+echo "Search for recent articles about the future of AI and write me a 500 word essay on the findings" | fabric --agents
+````
-This feature works with all OpenAI and Ollama models but does NOT work with Claude. You can specify your model with the -m flag.
-
-For more information about this amazing project, please visit https://github.com/MervinPraison/PraisonAI.
-
-## Helper Apps
-
-These are helper tools to work with Fabric. Examples include things like getting transcripts from media files, getting metadata about media, etc.
-
-## yt (YouTube)
-
-`yt` is a command that uses the YouTube API to pull transcripts, pull user comments, get video duration, and other functions. It's primary function is to get a transcript from a video that can then be stitched (piped) into other Fabric Patterns.
-
-```bash
-usage: yt [-h] [--duration] [--transcript] [url]
-
-vm (video meta) extracts metadata about a video, such as the transcript and the video's duration. By Daniel Miessler.
-
-positional arguments:
- url YouTube video URL
-
-options:
- -h, --help Show this help message and exit
- --duration Output only the duration
- --transcript Output only the transcript
- --comments Output only the user comments
-```
-
-## ts (Audio transcriptions)
-
-'ts' is a command that uses the OpenAI Whisper API to transcribe audio files. Due to the context window, this tool uses pydub to split the files into 10 minute segments. for more information on pydub, please refer https://github.com/jiaaro/pydub
-
-### Installation
-
-```bash
-
-mac:
-brew install ffmpeg
-
-linux:
-apt install ffmpeg
-
-windows:
-download instructions https://www.ffmpeg.org/download.html
-```
-
-```bash
-ts -h
-usage: ts [-h] audio_file
-
-Transcribe an audio file.
-
-positional arguments:
- audio_file The path to the audio file to be transcribed.
-
-options:
- -h, --help show this help message and exit
-```
-
-## Save
-
-`save` is a "tee-like" utility to pipeline saving of content, while keeping the output stream intact. Can optionally generate "frontmatter" for PKM utilities like Obsidian via the
-"FABRIC_FRONTMATTER" environment variable
-
-If you'd like to default variables, set them in `~/.config/fabric/.env`. `FABRIC_OUTPUT_PATH` needs to be set so `save` where to write. `FABRIC_FRONTMATTER_TAGS` is optional, but useful for tracking how tags have entered your PKM, if that's important to you.
-
-### usage
-
-```bash
-usage: save [-h] [-t, TAG] [-n] [-s] [stub]
-
-save: a "tee-like" utility to pipeline saving of content, while keeping the output stream intact. Can optionally generate "frontmatter" for PKM utilities like Obsidian via the
-"FABRIC_FRONTMATTER" environment variable
-
-positional arguments:
- stub stub to describe your content. Use quotes if you have spaces. Resulting format is YYYY-MM-DD-stub.md by default
-
-options:
- -h, --help show this help message and exit
- -t, TAG, --tag TAG add an additional frontmatter tag. Use this argument multiple timesfor multiple tags
- -n, --nofabric don't use the fabric tags, only use tags from --tag
- -s, --silent don't use STDOUT for output, only save to the file
-```
-
-### Example
-
-```bash
-echo test | save --tag extra-tag stub-for-name
-test
-
-$ cat ~/obsidian/Fabric/2024-03-02-stub-for-name.md
----
-generation_date: 2024-03-02 10:43
-tags: fabric-extraction stub-for-name extra-tag
----
-test
-```
+This feature works with all openai and ollama models but does NOT work with claude. You can specify your model with the -m flag
## Meta
> [!NOTE]
> Special thanks to the following people for their inspiration and contributions!
+- _Jonathan Dunn_ for all of his help with this project, including this new go verision, as well as the gui
- _Caleb Sima_ for pushing me over the edge of whether to make this a public project or not.
-- _Joel Parish_ for super useful input on the project's Github directory structure.
-- _Jonathan Dunn_ for spectacular work on the soon-to-be-released universal client.
+- _Joel Parish_ for super useful input on the project's Github directory structure..
- _Joseph Thacker_ for the idea of a `-c` context flag that adds pre-created context in the `./config/fabric/` directory to all Pattern queries.
- _Jason Haddix_ for the idea of a stitch (chained Pattern) to filter content using a local model before sending on to a cloud model, i.e., cleaning customer data using `llama2` before sending on to `gpt-4` for analysis.
- _Dani Goland_ for enhancing the Fabric Server (Mill) infrastructure by migrating to FastAPI, breaking the server into discrete pieces, and Dockerizing the entire thing.
diff --git a/cli/cli.go b/cli/cli.go
new file mode 100644
index 0000000..974b144
--- /dev/null
+++ b/cli/cli.go
@@ -0,0 +1,145 @@
+package cli
+
+import (
+ "fmt"
+ "os"
+ "path/filepath"
+ "strconv"
+
+ "github.com/danielmiessler/fabric/core"
+ "github.com/danielmiessler/fabric/db"
+)
+
+// Cli Controls the cli. It takes in the flags and runs the appropriate functions
+func Cli() (message string, err error) {
+ var currentFlags *Flags
+ if currentFlags, err = Init(); err != nil {
+ // we need to reset error, because we want to show double help messages
+ err = nil
+ return
+ }
+
+ var homedir string
+ if homedir, err = os.UserHomeDir(); err != nil {
+ return
+ }
+
+ db := db.NewDb(filepath.Join(homedir, ".config/fabric"))
+
+ // if the setup flag is set, run the setup function
+ if currentFlags.Setup {
+ _ = db.Configure()
+ _, err = Setup(db, currentFlags.SetupSkipUpdatePatterns)
+ return
+ }
+
+ var fabric *core.Fabric
+ if err = db.Configure(); err != nil {
+ fmt.Println("init is failed, run start the setup procedure", err)
+ if fabric, err = Setup(db, currentFlags.SetupSkipUpdatePatterns); err != nil {
+ return
+ }
+ } else {
+ if fabric, err = core.NewFabric(db); err != nil {
+ fmt.Println("fabric can't initialize, please run the --setup procedure", err)
+ return
+ }
+ }
+
+ // if the update patterns flag is set, run the update patterns function
+ if currentFlags.UpdatePatterns {
+ err = fabric.PopulateDB()
+ return
+ }
+
+ if currentFlags.ChangeDefaultModel {
+ err = fabric.SetupDefaultModel()
+ return
+ }
+
+ // if the latest patterns flag is set, run the latest patterns function
+ if currentFlags.LatestPatterns != "0" {
+ var parsedToInt int
+ if parsedToInt, err = strconv.Atoi(currentFlags.LatestPatterns); err != nil {
+ return
+ }
+
+ if err = db.Patterns.LatestPatterns(parsedToInt); err != nil {
+ return
+ }
+ return
+ }
+
+ // if the list patterns flag is set, run the list all patterns function
+ if currentFlags.ListPatterns {
+ err = db.Patterns.ListNames()
+ return
+ }
+
+ // if the list all models flag is set, run the list all models function
+ if currentFlags.ListAllModels {
+ fabric.GetModels().Print()
+ return
+ }
+
+ // if the list all contexts flag is set, run the list all contexts function
+ if currentFlags.ListAllContexts {
+ err = db.Contexts.ListNames()
+ return
+ }
+
+ // if the list all sessions flag is set, run the list all sessions function
+ if currentFlags.ListAllSessions {
+ err = db.Sessions.ListNames()
+ return
+ }
+
+ // if the interactive flag is set, run the interactive function
+ // if currentFlags.Interactive {
+ // interactive.Interactive()
+ // }
+
+ // if none of the above currentFlags are set, run the initiate chat function
+
+ var chatter *core.Chatter
+ if chatter, err = fabric.GetChatter(currentFlags.Model, currentFlags.Stream); err != nil {
+ return
+ }
+
+ if message, err = chatter.Send(currentFlags.BuildChatRequest(), currentFlags.BuildChatOptions()); err != nil {
+ return
+ }
+
+ if !currentFlags.Stream {
+ fmt.Println(message)
+ }
+
+ // if the copy flag is set, copy the message to the clipboard
+ if currentFlags.Copy {
+ if err = fabric.CopyToClipboard(message); err != nil {
+ return
+ }
+ }
+
+ // if the output flag is set, create an output file
+ if currentFlags.Output != "" {
+ err = fabric.CreateOutputFile(message, currentFlags.Output)
+ }
+ return
+}
+
+func Setup(db *db.Db, skipUpdatePatterns bool) (ret *core.Fabric, err error) {
+ ret = core.NewFabricForSetup(db)
+
+ if err = ret.Setup(); err != nil {
+ return
+ }
+
+ if !skipUpdatePatterns {
+ if err = ret.PopulateDB(); err != nil {
+ return
+ }
+ }
+
+ return
+}
diff --git a/cli/flags.go b/cli/flags.go
new file mode 100644
index 0000000..a168dc8
--- /dev/null
+++ b/cli/flags.go
@@ -0,0 +1,105 @@
+package cli
+
+import (
+ "bufio"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+
+ "github.com/danielmiessler/fabric/common"
+ "github.com/jessevdk/go-flags"
+)
+
+// Flags create flags struct. the users flags go into this, this will be passed to the chat struct in cli
+type Flags struct {
+ Pattern string `short:"p" long:"pattern" description:"Choose a pattern" default:""`
+ Context string `short:"C" long:"context" description:"Choose a context" default:""`
+ Session string `long:"session" description:"Choose a session"`
+ Setup bool `short:"S" long:"setup" description:"Run setup"`
+ SetupSkipUpdatePatterns bool `long:"setup-skip-update-patterns" description:"Skip update patterns at setup"`
+ Temperature float64 `short:"t" long:"temperature" description:"Set temperature" default:"0.7"`
+ TopP float64 `short:"T" long:"topp" description:"Set top P" default:"0.9"`
+ Stream bool `short:"s" long:"stream" description:"Stream"`
+ PresencePenalty float64 `short:"P" long:"presencepenalty" description:"Set presence penalty" default:"0.0"`
+ FrequencyPenalty float64 `short:"F" long:"frequencypenalty" description:"Set frequency penalty" default:"0.0"`
+ ListPatterns bool `short:"l" long:"listpatterns" description:"List all patterns"`
+ ListAllModels bool `short:"L" long:"listmodels" description:"List all available models"`
+ ListAllContexts bool `short:"x" long:"listcontexts" description:"List all contexts"`
+ ListAllSessions bool `short:"X" long:"listsessions" description:"List all sessions"`
+ UpdatePatterns bool `short:"U" long:"updatepatterns" description:"Update patterns"`
+ AddContext bool `short:"A" long:"addcontext" description:"Add a context"`
+ Message string `hidden:"true" description:"Message to send to chat"`
+ Copy bool `short:"c" long:"copy" description:"Copy to clipboard"`
+ Model string `short:"m" long:"model" description:"Choose model"`
+ Output string `short:"o" long:"output" description:"Output to file" default:""`
+ LatestPatterns string `short:"n" long:"latest" description:"Number of latest patterns to list" default:"0"`
+ ChangeDefaultModel bool `short:"d" long:"changeDefaultModel" description:"Change default pattern"`
+}
+
+// Init Initialize flags. returns a Flags struct and an error
+func Init() (ret *Flags, err error) {
+ var message string
+
+ ret = &Flags{}
+ parser := flags.NewParser(ret, flags.Default)
+ var args []string
+ if args, err = parser.Parse(); err != nil {
+ return
+ }
+
+ info, _ := os.Stdin.Stat()
+ hasStdin := (info.Mode() & os.ModeCharDevice) == 0
+
+ // takes input from stdin if it exists, otherwise takes input from args (the last argument)
+ if hasStdin {
+ if message, err = readStdin(); err != nil {
+ err = errors.New("error: could not read from stdin")
+ return
+ }
+ } else if len(args) > 0 {
+ message = args[len(args)-1]
+ } else {
+ message = ""
+ }
+ ret.Message = message
+
+ return
+}
+
+// readStdin reads from stdin and returns the input as a string or an error
+func readStdin() (string, error) {
+ reader := bufio.NewReader(os.Stdin)
+ var input string
+ for {
+ line, err := reader.ReadString('\n')
+ if err != nil {
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ return "", fmt.Errorf("error reading from stdin: %w", err)
+ }
+ input += line
+ }
+ return input, nil
+}
+
+func (o *Flags) BuildChatOptions() (ret *common.ChatOptions) {
+ ret = &common.ChatOptions{
+ Temperature: o.Temperature,
+ TopP: o.TopP,
+ PresencePenalty: o.PresencePenalty,
+ FrequencyPenalty: o.FrequencyPenalty,
+ }
+ return
+}
+
+func (o *Flags) BuildChatRequest() (ret *common.ChatRequest) {
+ ret = &common.ChatRequest{
+ ContextName: o.Context,
+ SessionName: o.Session,
+ PatternName: o.Pattern,
+ Message: o.Message,
+ }
+ return
+}
diff --git a/common/configurable.go b/common/configurable.go
new file mode 100644
index 0000000..0ec61d0
--- /dev/null
+++ b/common/configurable.go
@@ -0,0 +1,213 @@
+package common
+
+import (
+ "bytes"
+ "fmt"
+ "os"
+ "strings"
+)
+
+const AnswerReset = "reset"
+
+type Configurable struct {
+ Settings
+ SetupQuestions
+
+ Label string
+ EnvNamePrefix string
+
+ ConfigureCustom func() error
+}
+
+func (o *Configurable) GetName() string {
+ return o.Label
+}
+
+func (o *Configurable) GetSettings() Settings {
+ return o.Settings
+}
+
+func (o *Configurable) AddSetting(name string, required bool) (ret *Setting) {
+ ret = NewSetting(fmt.Sprintf("%v%v", o.EnvNamePrefix, BuildEnvVariable(name)), required)
+ o.Settings = append(o.Settings, ret)
+ return
+}
+
+func (o *Configurable) AddSetupQuestion(name string, required bool) (ret *SetupQuestion) {
+ return o.AddSetupQuestionCustom(name, required, "")
+}
+
+func (o *Configurable) AddSetupQuestionCustom(name string, required bool, question string) (ret *SetupQuestion) {
+ setting := o.AddSetting(name, required)
+ ret = &SetupQuestion{Setting: setting, Question: question}
+ if ret.Question == "" {
+ ret.Question = fmt.Sprintf("Enter your %v %v", o.Label, strings.ToUpper(name))
+ }
+ o.SetupQuestions = append(o.SetupQuestions, ret)
+ return
+}
+
+func (o *Configurable) Configure() (err error) {
+ if err = o.Settings.Configure(); err != nil {
+ return
+ }
+
+ if o.ConfigureCustom != nil {
+ err = o.ConfigureCustom()
+ }
+ return
+}
+
+func (o *Configurable) Setup() (err error) {
+ if err = o.Ask(o.Label); err != nil {
+ return
+ }
+
+ err = o.Configure()
+ return
+}
+
+func NewSetting(envVariable string, required bool) *Setting {
+ return &Setting{
+ EnvVariable: envVariable,
+ Required: required,
+ }
+}
+
+type Setting struct {
+ EnvVariable string
+ Value string
+ Required bool
+}
+
+func (o *Setting) IsValid() bool {
+ return o.IsDefined() || !o.Required
+}
+
+func (o *Setting) IsValidErr() (err error) {
+ if !o.IsValid() {
+ err = fmt.Errorf("%v=%v, is not valid", o.EnvVariable, o.Value)
+ }
+ return
+}
+
+func (o *Setting) IsDefined() bool {
+ return o.Value != ""
+}
+
+func (o *Setting) Configure() error {
+ if o.Value == "" {
+ o.Value = os.Getenv(o.EnvVariable)
+ }
+ return o.IsValidErr()
+}
+
+func (o *Setting) FillEnvFileContent(buffer *bytes.Buffer) {
+ if o.IsDefined() {
+ buffer.WriteString(o.EnvVariable)
+ buffer.WriteString("=")
+ //buffer.WriteString("\"")
+ buffer.WriteString(o.Value)
+ //buffer.WriteString("\"")
+ buffer.WriteString("\n")
+ }
+ return
+}
+
+func (o *Setting) Print() {
+ fmt.Printf("%v: %v\n", o.EnvVariable, o.Value)
+}
+
+type SetupQuestion struct {
+ *Setting
+ Question string
+}
+
+func (o *SetupQuestion) Ask(label string) (err error) {
+ var prefix string
+
+ if label != "" {
+ prefix = fmt.Sprintf("[%v] ", label)
+ } else {
+ prefix = ""
+ }
+
+ fmt.Println()
+ if o.Value != "" {
+ fmt.Printf("%v%v (leave empty for '%s' or type '%v' to remove the value):\n",
+ prefix, o.Question, o.Value, AnswerReset)
+ } else {
+ fmt.Printf("%v%v (leave empty to skip):\n", prefix, o.Question)
+ }
+
+ var answer string
+ fmt.Scanln(&answer)
+ answer = strings.TrimRight(answer, "\n")
+ if answer == "" {
+ answer = o.Value
+ } else if strings.ToLower(answer) == AnswerReset {
+ answer = ""
+ }
+ err = o.OnAnswer(answer)
+ return
+}
+
+func (o *SetupQuestion) OnAnswer(answer string) (err error) {
+ o.Value = answer
+ err = o.IsValidErr()
+ return
+}
+
+type Settings []*Setting
+
+func (o Settings) IsConfigured() (ret bool) {
+ ret = true
+ for _, setting := range o {
+ if ret = setting.IsValid(); !ret {
+ break
+ }
+ }
+ return
+}
+
+func (o Settings) Configure() (err error) {
+ for _, setting := range o {
+ if err = setting.Configure(); err != nil {
+ break
+ }
+ }
+ return
+}
+
+func (o Settings) FillEnvFileContent(buffer *bytes.Buffer) {
+ for _, setting := range o {
+ setting.FillEnvFileContent(buffer)
+ }
+ return
+}
+
+type SetupQuestions []*SetupQuestion
+
+func (o SetupQuestions) Ask(label string) (err error) {
+ fmt.Println()
+ fmt.Printf("[%v]\n", label)
+ for _, question := range o {
+ if err = question.Ask(""); err != nil {
+ break
+ }
+ }
+ return
+}
+
+func BuildEnvVariablePrefix(name string) (ret string) {
+ ret = BuildEnvVariable(name)
+ if ret != "" {
+ ret += "_"
+ }
+ return
+}
+
+func BuildEnvVariable(name string) string {
+ name = strings.TrimSpace(name)
+ return strings.ReplaceAll(strings.ToUpper(name), " ", "_")
+}
diff --git a/common/domain.go b/common/domain.go
new file mode 100644
index 0000000..b11458c
--- /dev/null
+++ b/common/domain.go
@@ -0,0 +1,21 @@
+package common
+
+type Message struct {
+ Role string `json:"role"`
+ Content string `json:"content"`
+}
+
+type ChatRequest struct {
+ ContextName string
+ SessionName string
+ PatternName string
+ Message string
+}
+
+type ChatOptions struct {
+ Model string
+ Temperature float64
+ TopP float64
+ PresencePenalty float64
+ FrequencyPenalty float64
+}
diff --git a/common/messages.go b/common/messages.go
new file mode 100644
index 0000000..dd1e330
--- /dev/null
+++ b/common/messages.go
@@ -0,0 +1,22 @@
+package common
+
+// NormalizeMessages remove empty messages and ensure messages order user-assist-user
+func NormalizeMessages(msgs []*Message, defaultUserMessage string) (ret []*Message) {
+ // Iterate over messages to enforce the odd position rule for user messages
+ fullMessageIndex := 0
+ for _, message := range msgs {
+ if message.Content == "" {
+ // Skip empty messages as the anthropic API doesn't accept them
+ continue
+ }
+
+ // Ensure, that each odd position shall be a user message
+ if fullMessageIndex%2 == 0 && message.Role != "user" {
+ ret = append(ret, &Message{Role: "user", Content: defaultUserMessage})
+ fullMessageIndex++
+ }
+ ret = append(ret, message)
+ fullMessageIndex++
+ }
+ return
+}
diff --git a/common/vendor.go b/common/vendor.go
new file mode 100644
index 0000000..d5c7aa0
--- /dev/null
+++ b/common/vendor.go
@@ -0,0 +1,12 @@
+package common
+
+type Vendor interface {
+ GetName() string
+ IsConfigured() bool
+ Configure() error
+ ListModels() ([]string, error)
+ SendStream([]*Message, *ChatOptions, chan string) error
+ Send([]*Message, *ChatOptions) (string, error)
+ GetSettings() Settings
+ Setup() error
+}
diff --git a/context/alma.md b/context/alma.md
deleted file mode 100644
index bbfd6e7..0000000
--- a/context/alma.md
+++ /dev/null
@@ -1,201 +0,0 @@
-## Document Purpose
-
-This document captures the SPQA policy and State for Alma Security, a security startup out of Redwood City, Ca.
-
-This is part of the SPQA context that will be used to answer questions and create artifacts for the company, e.g., company strategy, security strategy, quarterly security reports (QSRs), project plans, recommendations on which projects to undertake, which investments to take and avoid, and other such decisions.
-
-A major aspect of the SPQA system is the definition of the company's mission, goals, KPIs, and challenges. These shape everything within the company and thus should be used to shape the recommendations made when asked.
-
-In addition to the clearly stated goals and other defining characteristics listed above, there will also be a streaming list of updates coming into this system using the Activity document.
-
-Those will be changes, updates, or modifications to the direction of the company. For example, if Goal number 4 is to build a new datacenter in Boise, Idaho, but we see an update in the Activity section that says we've lost the ability to build in Boise, we should consider goal #4 out of the picture for prioritization and other decision purposes. In other words, the streaming activity log into this document should be considered updates to the core content.
-
-## Company History
-
-Alma Security was started by Chris Meyers, who was previously at Sigma Systems as CTO and HPE as a senior security engineer.
-
-He started the company because, "I saw a gap in the authentication market, where companies were only looking at one or two aspects of one's identity to do authentication. They we're looking at the whole picture and turning that into a continuous authentication story."
-
-## Company Mission
-
-The mission of Alma Security is to ensure businesses can continuously authenticate their users using their whole selves.
-
-## Company Goals (G1 means goal 1, G2 is goal 2, etc. Treat each item (goal/kpi/etc) as half as important as the one before it.)
-
-NOTE: Some goals are things like project rollouts which serve the higher goals. In that case they shouldn't always be considered so much lower priority because one is serving the other.
-
-## Company Goals
-
-- G1: Achieve 20% market share by January 2025
-- G2: Hit 10000 active customers by January 2025
-- G3: Hit a customer trust score of 90+% by January 2025
-- G4: Get churn below 5% by August 2024
-- G5: Launch in Europe by August 2024
-- G6: Launch in India by November 2024
-- G7: Launch Mood-monitor integration by February 2024
-- G8: Launch partnership with Apple Passkeys by June 2024
-
-## Company KPIs
-
-- K1: Current marketshare percentage
-- K2: Number of active customers
-- K3: Current churn percentage
-- K4: Launched_in_Europe (yes/no)
-- K4: Launched_in_India (yes/no)
-
------------------------------------------------------------------------------------------------------------------------
-
-## Security Team Mission
-
-- SM1: Protect Alma Security's customers and intellectual property from security and privacy incidents.
-
-## Security Team Goals
-
-- SG1: Secure all customer data -- especially biometric -- from security and privacy incidents.
-- SG2: Protect Alma Security's intellectual property from being captured by unauthorized parties.
-- SG3: Reach a time to detect malicious behavior of less than 4 minutes by January 2025
-- SG4: Ensure the public trusts our product, because it's an authentication product we can't survive if people don't trust us.
-- SG5: Reach a time to remediate critical vulnerabilities on crown jewel systems of less than 16 hours by August 2025
-- SG6: Reach a time to remediate critical vulnerabilities on all systems of less than 3 days by August 2025
-- SG7: Complete audit of Apple Passkey integration by February 2025
-- SG8: Complete remediation of Apple Passkey vulns by February 2025
-
-## Security Team KPIs (How we measure the team)
-
-- SK1: TTD: Time to detect malicious behavior (Minutes)
-- SK1: TTI: Time to begin investigation of malicious behavior (Minutes)
-- SK3: TTR-CJC: Time to remediate critical vulnerabilities on crown jewel systems (Hours)
-- SK3: TTR-C: Time to remediate critical vulnerabilities on all systems (Hours)
-- SK4: PT: Public trust score (Complete, Significant, Moderate, Minimal, Distrust, N/A)
-
-## Risk Register (The things we're most worried about)
-
-- R1: Our infrastructure security team is understaffed by 50% after 5 key people left
-- R2: We are not currently monitoring our external perimeter for attack surface related vulnerabilities like open ports, listening applications, unknown hosts, unknown subdomains pointing to these things, etc. We only do scans once every couple of months and we don't really have anyone to look at the results
-- R3: It takes us multiple days to investigate potential malicious behavior on our systems.
-- R4: We lack a full list of our assets, including externally facing hosts, S3 buckets, etc., which make up our attack surface
-- R5: We have a low public trust score due to the events of 2022.
-
-## Security Team Narrative
-
-### Background
-
-Alma hired a new security team starting in January of 2023 and we have been building out the program since then. The philosophy and approach for the security team is to explicitly articulate what we believe the highest risks are to Alma, to deploy targeted strategies to address those risks, and to use clear, transparent KPIs to show progress towards our goals over time.
-
-### Current Risks
-
-So our risk register looks like this:
-
-1. We are understaffed by 50% after 5 key people left in 2022
-2. Our perimeter is not being monitored for attack surface related vulnerabilities
-3. It takes us too long to detect and start investigating malicious behavior on our systems
-4. We do not have a full list of our assets, which makes it difficult to know what we need to protect
-5. We have a low public trust score due to the events of 2022
-
-### Strategies
-
-As such, our strategies are as follows:
-
-1. Hire 5 more A-tier security professionals
-2. Purchase and implement an attack surface management solution
-3. Invest in our detection and response capabilities
-4. Purchase an asset inventory system that integrates with our attack surface management tool
-5. Leverage PR to share as much of our progress as possible with the public to rebuild trust
-
-### How We're Doing
-
-We believe being transparent about our progress is key to everything, and for that reason we maintain a limited number of KPIs that we update every quarter. These metrics will not change often. They will remain consistent so that it's easy to track how we're spending our resources and the progress we're making.
-
-Those KPIs are:
-
-1. Time to detect malicious behavior
-2. Time to start investigating malicious behavior
-3. Time to remediate critical vulnerabilities on crown jewel systems
-4. Time to remediate critical vulnerabilities on all systems
-5. Our public trust score
-
-As of $DATE$, our KPIs for these are currently:
-
-$GIVE CURRENT KPIs from the Activity section below$
-
-$INSERT GRAPHS OF KPI PROGRESS OVER TIME HERE$
-
-## Security Team Strategies
-
-- STS1: Hire 5 more A-tier security professionals
-- STS2: Purchase an attack surface management solution
-- STS3: Invest in our detection and response capabilities
-- STS4: Purchase an asset inventory system that integrates with our attack surface management tool
-
-## Infrastructure Notes (a basic description of our tech stack and various context around it)
-
-- We currently have no WAF protecting our main web app but we're considering adding one
-- We have had some issues with S3 buckets becoming public, or being set up as public, which has lead to some close calls with customer data almost being exposed.
-- alma.amazon-domain.com is our primary S3 bucket that contains everything, but it's not public readable or listable
-- We have a root account for our AWS account that doesn't yet have 2FA on it, but we're working on fixing that within a few weeks (but it's been open for a few months)
-- We also use Postgres for all our databases.
-- Developers have root access to the all kubernetes nodes via SSH on port 45,001 using a shared developer key issued during laptop provisioning.
-- We're a kubernetes shop and do everything through AWS
-- We're logging most stuff to Cloudtrail and we kind of use guarduty, but we don't have a 24/7 team to monitor alerts and logs. We should add that to our list of challenges next time we update our overarll policy
-- We also have a Windows infrastructure because some key personell came from Microsoft. The DC is hosted in our head office which is in Redwood City, and anyone who works in that office (most of the 300 employees) uses that to log in when tehy start work. The domain is ALMA.
-- There's a domain-joined fileserver running Windows 2012 that most people use to upload new ideas and plans for new products. It uses Windows authentication from the domain.
-- We use a palo alto firewall with 2fa using windows authenticator tied to SSO.
-- The name of the AI system doing all this context creation using SPQA is Alma, which is also the name of the company.
-- We use Workday for HR stuff. Slack for realtime communications. Outlook 365 as a service. Sentinel One on the workstations and laptops. Servers in AWS are mostly Amazon Linux 2 with a few Ubuntu boxes that are a few years old.
-- We also primarily use Postgres for all of our systems.
-
-## Team
-
-TEAM MEMBER | TEAM ASSIGNED | SKILLS | PAY LEVEL | LOCATION | PROJECTS
-
-Nadia Khan | Detection and Response | D&R (Expert), AWS (Strong), Python (Expert), Kubernetes (Basic), Postgres (Basic) | $249K | Redwood City
-Chris Magann | Vulnerability Management | VM (Expert), AWS (Strong), Python (Basic), Postgres (Basic) | $212K | Redwood City
-Tigan Wang | Vulnerability Management | VM (Expert), AWS (Strong), Python (Basic), Postgres (Basic) | $217K | Redwood City
-
-## Projects
-
-PROJECT NAME | PROJECT DESCRIPTION | PROJECT PRIORITY | PROJECT MEMBERS | START DATE | END DATE | STATUS | PROJECT COST
-
-WAF Install | Install a WAF in front of our main web app | Critical | Nadia Khan | 2024-01-01 - Ongoing | In Progress | $112K one-time, $9K/month
-
-Multi-Factor Authentication (MFA) Rollout | Implement MFA across all internal and external systems | Critical | Chris Magaan | 2024-01-15 | 2024-05-01 | Planned | $80K one-time, $5K/month
-
-Procure and Implement ASM | Implement continuous monitoring for attack surface vulnerabilities | High | Tigan Wang | 2024-02-15 | 2024-06-15 | Not Started | $75K one-time, $6K/month
-
-Data Encryption Upgrade | Upgrade encryption protocols for all sensitive data | Medium | Nadia Khan | 2024-04-01 | 2024-08-01 | Planned | $95K one-time
-
-Incident Response Enhancement | Develop and implement a 24/7 incident response team | High | Nadia Khan | 2024-03-01 | 2024-07-01 | In Progress | $150K one-time, $10K/month
-
-Cloud Security Optimization | Optimize AWS cloud security configurations and practices | Medium | Tigan Wang | 2024-02-01 | 2024-06-01 | In Progress | $100K one-time, $8K/month
-
-S3 Bucket Security | Review and secure all S3 buckets to prevent data breaches | High | Chris Magaan | 2024-01-10 | 2024-04-10 | In Progress | $70K one-time, $5K/month
-
-SQL Injection Mitigation | Implement measures to eliminate SQL injection vulnerabilities | High | Tigan Wang | 2024-01-20 | 2024-05-20 | Not Started | $60K one-time
-
-## CURRENT STATE (KPIs, Metrics, Project Activity Updates, etc.)
-- October 2022: Current time to detect malicious behavior is 81 hours
-- October 2022: Current time to start investigating malicious behavior is 82 hours
-- October 2022: Current time to remediate critical vulnerabilities on crown jewel systems is 21 days
-- October 2022: Current time to remediate critical vulnerabilities on all systems is 51 days
-- January 2023: Current time to detect malicious behavior is 62 hours
-- January 2023: Current time to start investigating malicious behavior is 72 hours
-- January 2023: Current time to remediate critical vulnerabilities on crown jewel systems is 17 days
-- January 2023: Current time to remediate critical vulnerabilities on all systems is 43 days
-- July 2023: Current time to detect malicious behavior is 29 hours
-- July 2023: Current time to start investigating malicious behavior is 41 hours
-- July 2023: Current time to remediate critical vulnerabilities on crown jewel systems is 12 days
-- July 2023: Current time to remediate critical vulnerabilities on all systems is 29 days
-- November 2023: Current time to start detect malicious behavior is 12 hours
-- November 2023: Current time to start investigating malicious behavior is 16 hours
-- November 2023: Current time to remediate critical vulnerabilities on crown jewel systems is 9 days
-- November 2023: Current time to remediate critical vulnerabilities on all systems is 17 days
-- February 2024: Started attack surface management vendor selection process
-- January 2024: Current time to start detect malicious behavior is 9 hours
-- January 2024: Current time to start investigating malicious behavior is 14 hours
-- January 2024: Current time to remediate critical vulnerabilities on crown jewel systems is 8 days
-- January 2024: Current time to remediate critical vulnerabilities on all systems is 12 days
-- March 2024: We're now remediating crits on crown jewels in less than 6 days
-- April 2024: We're now remediating all criticals within 11 days
-- July 2024: Criticals are now being fixed in 9 days
-- August 1 2024: Criticals are now being fixed in 7 days
-- August 7 2024: Criticals are now being fixed in 6 days
diff --git a/core/chatter.go b/core/chatter.go
new file mode 100644
index 0000000..34b48cf
--- /dev/null
+++ b/core/chatter.go
@@ -0,0 +1,103 @@
+package core
+
+import (
+ "fmt"
+
+ "github.com/danielmiessler/fabric/common"
+ "github.com/danielmiessler/fabric/db"
+)
+
+type Chatter struct {
+ db *db.Db
+
+ Stream bool
+
+ model string
+ vendor common.Vendor
+}
+
+func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (message string, err error) {
+ var chatRequest *Chat
+ if chatRequest, err = o.NewChat(request); err != nil {
+ return
+ }
+
+ var messages []*common.Message
+ if messages, err = chatRequest.BuildMessages(); err != nil {
+ return
+ }
+
+ if opts.Model == "" {
+ opts.Model = o.model
+ }
+
+ if o.Stream {
+ channel := make(chan string)
+ go func() {
+ if streamErr := o.vendor.SendStream(messages, opts, channel); streamErr != nil {
+ channel <- streamErr.Error()
+ }
+ }()
+
+ for response := range channel {
+ message += response
+ fmt.Print(response)
+ }
+ } else {
+ if message, err = o.vendor.Send(messages, opts); err != nil {
+ return
+ }
+ }
+
+ if chatRequest.Session != nil && message != "" {
+ chatRequest.Session.Append(
+ &common.Message{Role: "system", Content: message},
+ &common.Message{Role: "user", Content: chatRequest.Message})
+ err = chatRequest.Session.Save()
+ }
+ return
+}
+
+func (o *Chatter) NewChat(request *common.ChatRequest) (ret *Chat, err error) {
+ ret = &Chat{}
+
+ if request.ContextName != "" {
+ var ctx *db.Context
+ if ctx, err = o.db.Contexts.LoadContext(request.ContextName); err != nil {
+ err = fmt.Errorf("could not find context %s: %v", request.ContextName, err)
+ return
+ }
+ ret.Context = ctx.Content
+ }
+
+ if request.SessionName != "" {
+ var sess *db.Session
+ if sess, err = o.db.Sessions.LoadOrCreateSession(request.SessionName); err != nil {
+ err = fmt.Errorf("could not find session %s: %v", request.SessionName, err)
+ return
+ }
+ ret.Session = sess
+ }
+
+ if request.PatternName != "" {
+ var pattern *db.Pattern
+ if pattern, err = o.db.Patterns.GetByName(request.PatternName); err != nil {
+ err = fmt.Errorf("could not find pattern %s: %v", request.PatternName, err)
+ return
+ }
+
+ if pattern.Pattern != "" {
+ ret.Pattern = pattern.Pattern
+ }
+ }
+
+ ret.Message = request.Message
+ return
+}
+
+type Chat struct {
+ Context string
+ Pattern string
+ Message string
+ Session *db.Session
+}
diff --git a/core/fabric.go b/core/fabric.go
new file mode 100644
index 0000000..cd85181
--- /dev/null
+++ b/core/fabric.go
@@ -0,0 +1,242 @@
+package core
+
+import (
+ "bytes"
+ "fmt"
+ "os"
+ "strconv"
+ "strings"
+
+ "github.com/atotto/clipboard"
+ "github.com/danielmiessler/fabric/common"
+ "github.com/danielmiessler/fabric/db"
+ "github.com/danielmiessler/fabric/vendors/anthropic"
+ "github.com/danielmiessler/fabric/vendors/azure"
+ "github.com/danielmiessler/fabric/vendors/gemini"
+ "github.com/danielmiessler/fabric/vendors/grocq"
+ "github.com/danielmiessler/fabric/vendors/ollama"
+ "github.com/danielmiessler/fabric/vendors/openai"
+ "github.com/pkg/errors"
+)
+
+const (
+ DefaultPatternsGitRepoUrl = "https://github.com/danielmiessler/fabric.git"
+ DefaultPatternsGitRepoFolder = "patterns"
+)
+
+func NewFabric(db *db.Db) (ret *Fabric, err error) {
+ ret = NewFabricBase(db)
+ err = ret.Configure()
+ return
+}
+
+func NewFabricForSetup(db *db.Db) (ret *Fabric) {
+ ret = NewFabricBase(db)
+ _ = ret.Configure()
+ return
+}
+
+// NewFabricBase Create a new Fabric from a list of already configured VendorsController
+func NewFabricBase(db *db.Db) (ret *Fabric) {
+ ret = &Fabric{
+ Db: db,
+ VendorsController: NewVendors(),
+ PatternsLoader: NewPatternsLoader(db.Patterns),
+ }
+
+ label := "Default"
+ ret.Configurable = &common.Configurable{
+ Label: label,
+ EnvNamePrefix: common.BuildEnvVariablePrefix(label),
+ ConfigureCustom: ret.configure,
+ }
+
+ ret.DefaultVendor = ret.AddSetting("Vendor", true)
+ ret.DefaultModel = ret.AddSetupQuestionCustom("Model", true,
+ "Enter the index the name of your default model")
+
+ ret.AddVendors(openai.NewClient(), azure.NewClient(), ollama.NewClient(), grocq.NewClient(),
+ gemini.NewClient(), anthropic.NewClient())
+
+ return
+}
+
+type Fabric struct {
+ *common.Configurable
+ *VendorsController
+ *PatternsLoader
+
+ Db *db.Db
+
+ DefaultVendor *common.Setting
+ DefaultModel *common.SetupQuestion
+}
+
+type ChannelName struct {
+ channel chan []string
+ name string
+}
+
+func (o *Fabric) SaveEnvFile() (err error) {
+ // Now create the .env with all configured VendorsController info
+ var envFileContent bytes.Buffer
+
+ o.Settings.FillEnvFileContent(&envFileContent)
+ o.PatternsLoader.FillEnvFileContent(&envFileContent)
+
+ for _, vendor := range o.Configured {
+ vendor.GetSettings().FillEnvFileContent(&envFileContent)
+ }
+
+ err = o.Db.SaveEnv(envFileContent.String())
+ return
+}
+
+func (o *Fabric) Setup() (err error) {
+ if err = o.SetupVendors(); err != nil {
+ return
+ }
+
+ if err = o.SetupDefaultModel(); err != nil {
+ return
+ }
+
+ if err = o.PatternsLoader.Setup(); err != nil {
+ return
+ }
+
+ err = o.SaveEnvFile()
+
+ return
+}
+
+func (o *Fabric) SetupDefaultModel() (err error) {
+ vendorsModels := o.GetModels()
+
+ vendorsModels.Print()
+
+ if err = o.Ask(o.Label); err != nil {
+ return
+ }
+
+ index, parseErr := strconv.Atoi(o.DefaultModel.Value)
+ if parseErr == nil {
+ o.DefaultVendor.Value, o.DefaultModel.Value = vendorsModels.GetVendorAndModelByModelIndex(index)
+ } else {
+ o.DefaultVendor.Value = vendorsModels.FindVendorsByModelFirst(o.DefaultModel.Value)
+ }
+
+ // verify
+ vendorNames := vendorsModels.FindVendorsByModel(o.DefaultModel.Value)
+ if len(vendorNames) == 0 {
+ err = errors.Errorf("You need to chose an available default model.")
+ return
+ }
+
+ fmt.Println()
+ o.DefaultVendor.Print()
+ o.DefaultModel.Print()
+
+ err = o.SaveEnvFile()
+
+ return
+}
+
+func (o *Fabric) SetupVendors() (err error) {
+ o.ResetConfigured()
+
+ for _, vendor := range o.All {
+ fmt.Println()
+ if vendorErr := vendor.Setup(); vendorErr == nil {
+ fmt.Printf("[%v] configured\n", vendor.GetName())
+ o.AddVendorConfigured(vendor)
+ } else {
+ fmt.Printf("[%v] skiped\n", vendor.GetName())
+ }
+ }
+
+ if !o.HasConfiguredVendors() {
+ err = errors.New("No vendors configured")
+ return
+ }
+
+ err = o.SaveEnvFile()
+
+ return
+}
+
+// Configure buildClient VendorsController based on the environment variables
+func (o *Fabric) configure() (err error) {
+ for _, vendor := range o.All {
+ if vendorErr := vendor.Configure(); vendorErr == nil {
+ o.AddVendorConfigured(vendor)
+ }
+ }
+ err = o.PatternsLoader.Configure()
+ return
+}
+
+func (o *Fabric) GetChatter(model string, stream bool) (ret *Chatter, err error) {
+ ret = &Chatter{
+ db: o.Db,
+ Stream: stream,
+ }
+
+ if model == "" {
+ ret.vendor = o.FindByName(o.DefaultVendor.Value)
+ ret.model = o.DefaultModel.Value
+ } else {
+ ret.vendor = o.FindByName(o.GetModels().FindVendorsByModelFirst(model))
+ ret.model = model
+ }
+
+ if ret.vendor == nil {
+ err = fmt.Errorf(
+ "could not find vendor.\n Model = %s\n DefaultModel = %s\n DefaultVendor = %s",
+ model, o.DefaultModel.Value, o.DefaultVendor.Value)
+ return
+ }
+ return
+}
+
+func (o *Fabric) CopyToClipboard(message string) (err error) {
+ if err = clipboard.WriteAll(message); err != nil {
+ err = fmt.Errorf("could not copy to clipboard: %v", err)
+ }
+ return
+}
+
+func (o *Fabric) CreateOutputFile(message string, fileName string) (err error) {
+ var file *os.File
+ if file, err = os.Create(fileName); err != nil {
+ err = fmt.Errorf("error creating file: %v", err)
+ return
+ }
+ defer file.Close()
+ if _, err = file.WriteString(message); err != nil {
+ err = fmt.Errorf("error writing to file: %v", err)
+ }
+ return
+}
+
+func (o *Chat) BuildMessages() (ret []*common.Message, err error) {
+ if o.Session != nil && len(o.Session.Messages) > 0 {
+ ret = append(ret, o.Session.Messages...)
+ }
+
+ systemMessage := strings.TrimSpace(o.Context) + strings.TrimSpace(o.Pattern)
+
+ if systemMessage != "" {
+ ret = append(ret, &common.Message{Role: "system", Content: systemMessage})
+ }
+
+ userMessage := strings.TrimSpace(o.Message)
+ if userMessage != "" {
+ ret = append(ret, &common.Message{Role: "user", Content: userMessage})
+ }
+
+ if ret == nil {
+ err = fmt.Errorf("no session, pattern or user messages provided")
+ }
+ return
+}
diff --git a/core/models.go b/core/models.go
new file mode 100644
index 0000000..980508e
--- /dev/null
+++ b/core/models.go
@@ -0,0 +1,97 @@
+package core
+
+import (
+ "fmt"
+ "sort"
+)
+
+func NewVendorsModels() *VendorsModels {
+ return &VendorsModels{VendorsModels: make(map[string][]string)}
+}
+
+type VendorsModels struct {
+ Vendors []string
+ VendorsModels map[string][]string
+ Errs []error
+}
+
+func (o *VendorsModels) AddVendorModels(vendor string, models []string) {
+ o.Vendors = append(o.Vendors, vendor)
+ o.VendorsModels[vendor] = models
+}
+
+func (o *VendorsModels) GetVendorAndModelByModelIndex(modelIndex int) (vendor string, model string) {
+ vendorModelIndexFrom := 0
+ vendorModelIndexTo := 0
+ for _, currenVendor := range o.Vendors {
+ vendorModelIndexFrom = vendorModelIndexTo + 1
+ vendorModelIndexTo = vendorModelIndexFrom + len(o.VendorsModels[currenVendor]) - 1
+
+ if modelIndex >= vendorModelIndexFrom && modelIndex <= vendorModelIndexTo {
+ vendor = currenVendor
+ model = o.VendorsModels[currenVendor][modelIndex-vendorModelIndexFrom]
+ break
+ }
+ }
+ return
+}
+
+func (o *VendorsModels) AddError(err error) {
+ o.Errs = append(o.Errs, err)
+}
+
+func (o *VendorsModels) Print() {
+ fmt.Printf("\nAvailable vendor models:\n")
+
+ sort.Strings(o.Vendors)
+
+ var currentModelIndex int
+ for _, vendor := range o.Vendors {
+ fmt.Println()
+ fmt.Printf("%s\n", vendor)
+ fmt.Println()
+ currentModelIndex = o.PrintVendor(vendor, currentModelIndex)
+ }
+ return
+}
+
+func (o *VendorsModels) PrintVendor(vendor string, modelIndex int) (currentModelIndex int) {
+ currentModelIndex = modelIndex
+ models := o.VendorsModels[vendor]
+ for _, model := range models {
+ currentModelIndex++
+ fmt.Printf("\t[%d]\t%s\n", currentModelIndex, model)
+ }
+ fmt.Println()
+ return
+}
+
+func (o *VendorsModels) GetVendorModels(vendor string) (models []string) {
+ models = o.VendorsModels[vendor]
+ return
+}
+
+func (o *VendorsModels) HasVendor(vendor string) (ret bool) {
+ ret = o.VendorsModels[vendor] != nil
+ return
+}
+
+func (o *VendorsModels) FindVendorsByModelFirst(model string) (ret string) {
+ vendors := o.FindVendorsByModel(model)
+ if len(vendors) > 0 {
+ ret = vendors[0]
+ }
+ return
+}
+
+func (o *VendorsModels) FindVendorsByModel(model string) (vendors []string) {
+ for vendor, models := range o.VendorsModels {
+ for _, m := range models {
+ if m == model {
+ vendors = append(vendors, vendor)
+ continue
+ }
+ }
+ }
+ return
+}
diff --git a/core/patterns_loader.go b/core/patterns_loader.go
new file mode 100644
index 0000000..0f77ea6
--- /dev/null
+++ b/core/patterns_loader.go
@@ -0,0 +1,275 @@
+package core
+
+import (
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+ "sort"
+ "strings"
+
+ "github.com/danielmiessler/fabric/common"
+ "github.com/danielmiessler/fabric/db"
+ "github.com/go-git/go-git/v5"
+ "github.com/go-git/go-git/v5/plumbing"
+ "github.com/go-git/go-git/v5/plumbing/object"
+ "github.com/go-git/go-git/v5/storage/memory"
+ "github.com/otiai10/copy"
+)
+
+func NewPatternsLoader(patterns *db.Patterns) (ret *PatternsLoader) {
+ label := "Patterns Loader"
+ ret = &PatternsLoader{
+ Patterns: patterns,
+ }
+
+ ret.Configurable = &common.Configurable{
+ Label: label,
+ EnvNamePrefix: common.BuildEnvVariablePrefix(label),
+ ConfigureCustom: ret.configure,
+ }
+
+ ret.DefaultGitRepoUrl = ret.AddSetupQuestionCustom("Git Repo Url", true,
+ "Enter the default Git repository URL for the patterns")
+ ret.DefaultGitRepoUrl.Value = DefaultPatternsGitRepoUrl
+
+ ret.DefaultFolder = ret.AddSetupQuestionCustom("Git Repo Patterns Folder", true,
+ "Enter the default folder in the Git repository where patterns are stored")
+ ret.DefaultFolder.Value = DefaultPatternsGitRepoFolder
+
+ return
+}
+
+type PatternsLoader struct {
+ *common.Configurable
+ Patterns *db.Patterns
+
+ DefaultGitRepoUrl *common.SetupQuestion
+ DefaultFolder *common.SetupQuestion
+
+ pathPatternsPrefix string
+ tempPatternsFolder string
+}
+
+func (o *PatternsLoader) configure() (err error) {
+ o.pathPatternsPrefix = fmt.Sprintf("%v/", o.DefaultFolder.Value)
+ o.tempPatternsFolder = filepath.Join(os.TempDir(), o.DefaultFolder.Value)
+
+ return
+}
+
+// PopulateDB downloads patterns from the internet and populates the patterns folder
+func (o *PatternsLoader) PopulateDB() (err error) {
+ fmt.Printf("Downloading patterns and Populating %s..\n", o.Patterns.Dir)
+ fmt.Println()
+ if err = o.gitCloneAndCopy(); err != nil {
+ return
+ }
+
+ if err = o.movePatterns(); err != nil {
+ return
+ }
+ return
+}
+
+// PersistPatterns copies custom patterns to the updated patterns directory
+func (o *PatternsLoader) PersistPatterns() (err error) {
+ var currentPatterns []os.DirEntry
+ if currentPatterns, err = os.ReadDir(o.Patterns.Dir); err != nil {
+ return
+ }
+
+ newPatternsFolder := o.tempPatternsFolder
+ var newPatterns []os.DirEntry
+ if newPatterns, err = os.ReadDir(newPatternsFolder); err != nil {
+ return
+ }
+
+ for _, currentPattern := range currentPatterns {
+ for _, newPattern := range newPatterns {
+ if currentPattern.Name() == newPattern.Name() {
+ break
+ }
+ copy.Copy(filepath.Join(o.Patterns.Dir, newPattern.Name()), filepath.Join(newPatternsFolder, newPattern.Name()))
+ }
+ }
+ return
+}
+
+// movePatterns copies the new patterns into the config directory
+func (o *PatternsLoader) movePatterns() (err error) {
+ os.MkdirAll(o.Patterns.Dir, os.ModePerm)
+
+ patternsDir := o.tempPatternsFolder
+ if err = o.PersistPatterns(); err != nil {
+ return
+ }
+
+ copy.Copy(patternsDir, o.Patterns.Dir) // copies the patterns to the config directory
+ err = os.RemoveAll(patternsDir)
+ return
+}
+
+// checks if a pattern already exists in the directory
+// func DoesPatternExistAlready(name string) (bool, error) {
+// entry := db.Entry{
+// Label: name,
+// }
+// _, err := entry.GetByName()
+// if err != nil {
+// return false, err
+// }
+// return true, nil
+// }
+
+func (o *PatternsLoader) gitCloneAndCopy() (err error) {
+ // Clones the given repository, creating the remote, the local branches
+ // and fetching the objects, everything in memory:
+ var r *git.Repository
+ if r, err = git.Clone(memory.NewStorage(), nil, &git.CloneOptions{
+ URL: o.DefaultGitRepoUrl.Value,
+ }); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ // ... retrieves the branch pointed by HEAD
+ var ref *plumbing.Reference
+ if ref, err = r.Head(); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ // ... retrieves the commit history for /patterns folder
+ var cIter object.CommitIter
+ if cIter, err = r.Log(&git.LogOptions{
+ From: ref.Hash(),
+ PathFilter: func(path string) bool {
+ return path == o.DefaultFolder.Value || strings.HasPrefix(path, o.pathPatternsPrefix)
+ },
+ }); err != nil {
+ fmt.Println(err)
+ return err
+ }
+
+ var changes []db.DirectoryChange
+ // ... iterates over the commits
+ if err = cIter.ForEach(func(c *object.Commit) (err error) {
+ // Get the files changed in this commit by comparing with its parents
+ parentIter := c.Parents()
+ if err = parentIter.ForEach(func(parent *object.Commit) (err error) {
+ var patch *object.Patch
+ if patch, err = parent.Patch(c); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ for _, fileStat := range patch.Stats() {
+ if strings.HasPrefix(fileStat.Name, o.pathPatternsPrefix) {
+ dir := filepath.Dir(fileStat.Name)
+ changes = append(changes, db.DirectoryChange{Dir: dir, Timestamp: c.Committer.When})
+ }
+ }
+ return
+ }); err != nil {
+ fmt.Println(err)
+ return
+ }
+ return
+ }); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ // Sort changes by timestamp
+ sort.Slice(changes, func(i, j int) bool {
+ return changes[i].Timestamp.Before(changes[j].Timestamp)
+ })
+
+ o.makeUniqueList(changes)
+
+ var commit *object.Commit
+ if commit, err = r.CommitObject(ref.Hash()); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ var tree *object.Tree
+ if tree, err = commit.Tree(); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ if err = tree.Files().ForEach(func(f *object.File) (err error) {
+ if strings.HasPrefix(f.Name, o.pathPatternsPrefix) {
+ // Create the local file path
+ localPath := filepath.Join(os.TempDir(), f.Name)
+
+ // Create the directories if they don't exist
+ if err = os.MkdirAll(filepath.Dir(localPath), os.ModePerm); err != nil {
+ fmt.Println(err)
+ return
+ }
+
+ // Write the file to the local filesystem
+ var blob *object.Blob
+ if blob, err = r.BlobObject(f.Hash); err != nil {
+ fmt.Println(err)
+ return
+ }
+ err = o.writeBlobToFile(blob, localPath)
+ return
+ }
+
+ return
+ }); err != nil {
+ fmt.Println(err)
+ }
+
+ return
+}
+
+func (o *PatternsLoader) writeBlobToFile(blob *object.Blob, path string) (err error) {
+ var reader io.ReadCloser
+ if reader, err = blob.Reader(); err != nil {
+ return
+ }
+ defer reader.Close()
+
+ // Create the file
+ var file *os.File
+ if file, err = os.Create(path); err != nil {
+ return
+ }
+ defer file.Close()
+
+ // Copy the contents of the blob to the file
+ if _, err = io.Copy(file, reader); err != nil {
+ return
+ }
+ return
+}
+
+func (o *PatternsLoader) makeUniqueList(changes []db.DirectoryChange) {
+ uniqueItems := make(map[string]bool)
+ for _, change := range changes {
+ if strings.TrimSpace(change.Dir) != "" && !strings.Contains(change.Dir, "=>") {
+ pattern := strings.ReplaceAll(change.Dir, o.pathPatternsPrefix, "")
+ pattern = strings.TrimSpace(pattern)
+ uniqueItems[pattern] = true
+ }
+ }
+
+ finalList := make([]string, 0, len(uniqueItems))
+ for _, change := range changes {
+ pattern := strings.ReplaceAll(change.Dir, o.pathPatternsPrefix, "")
+ pattern = strings.TrimSpace(pattern)
+ if _, exists := uniqueItems[pattern]; exists {
+ finalList = append(finalList, pattern)
+ delete(uniqueItems, pattern) // Remove to avoid duplicates in the final list
+ }
+ }
+
+ joined := strings.Join(finalList, "\n")
+ os.WriteFile(o.Patterns.UniquePatternsFilePath, []byte(joined), 0o644)
+}
diff --git a/core/vendors.go b/core/vendors.go
new file mode 100644
index 0000000..6c78b20
--- /dev/null
+++ b/core/vendors.go
@@ -0,0 +1,108 @@
+package core
+
+import (
+ "fmt"
+ "sync"
+
+ "github.com/danielmiessler/fabric/common"
+)
+
+func NewVendors() (ret *VendorsController) {
+ ret = &VendorsController{
+ All: map[string]common.Vendor{},
+ Configured: map[string]common.Vendor{},
+ }
+ return
+}
+
+type VendorsController struct {
+ All map[string]common.Vendor
+ Configured map[string]common.Vendor
+
+ Models *VendorsModels
+}
+
+func (o *VendorsController) AddVendors(vendors ...common.Vendor) {
+ for _, vendor := range vendors {
+ o.All[vendor.GetName()] = vendor
+ }
+}
+
+func (o *VendorsController) AddVendorConfigured(vendor common.Vendor) {
+ o.Configured[vendor.GetName()] = vendor
+}
+
+func (o *VendorsController) ResetConfigured() {
+ o.Configured = map[string]common.Vendor{}
+ o.Models = nil
+ return
+}
+
+func (o *VendorsController) GetModels() (ret *VendorsModels) {
+ if o.Models == nil {
+ o.readModels()
+ }
+ ret = o.Models
+ return
+}
+
+func (o *VendorsController) HasConfiguredVendors() bool {
+ return len(o.Configured) > 0
+}
+
+func (o *VendorsController) readModels() {
+ o.Models = NewVendorsModels()
+
+ var wg sync.WaitGroup
+ var channels []ChannelName
+
+ errorsChan := make(chan error, 3)
+
+ for _, vendor := range o.Configured {
+ // For each vendor:
+ // - Create a channel to collect output from the vendor model's list
+ // - Create a goroutine to query the vendor on its model
+ cn := ChannelName{channel: make(chan []string, 1), name: vendor.GetName()}
+ channels = append(channels, cn)
+ o.createGoroutine(&wg, vendor, cn, errorsChan)
+ }
+
+ // Let's wait for completion
+ wg.Wait() // Wait for all goroutines to finish
+ close(errorsChan)
+
+ for err := range errorsChan {
+ fmt.Println(err)
+ o.Models.AddError(err)
+ }
+
+ // And collect output
+ for _, cn := range channels {
+ models := <-cn.channel
+ if models != nil {
+ o.Models.AddVendorModels(cn.name, models)
+ }
+ }
+ return
+}
+
+func (o *VendorsController) FindByName(name string) (ret common.Vendor) {
+ ret = o.Configured[name]
+ return
+}
+
+// Create a goroutine to list models for the given vendor
+func (o *VendorsController) createGoroutine(wg *sync.WaitGroup, vendor common.Vendor, cn ChannelName, errorsChan chan error) {
+ wg.Add(1)
+
+ go func() {
+ defer wg.Done()
+ models, err := vendor.ListModels()
+ if err != nil {
+ errorsChan <- err
+ cn.channel <- nil
+ } else {
+ cn.channel <- models
+ }
+ }()
+}
diff --git a/db/chroma.sqlite3 b/db/chroma.sqlite3
deleted file mode 100644
index 0ff30fc..0000000
Binary files a/db/chroma.sqlite3 and /dev/null differ
diff --git a/db/contexts.go b/db/contexts.go
new file mode 100644
index 0000000..ecc8907
--- /dev/null
+++ b/db/contexts.go
@@ -0,0 +1,35 @@
+package db
+
+import (
+ "os"
+)
+
+type Contexts struct {
+ *Storage
+}
+
+// LoadContext Load a context from file
+func (o *Contexts) LoadContext(name string) (ret *Context, err error) {
+ path := o.BuildFilePathByName(name)
+
+ var content []byte
+ if content, err = os.ReadFile(path); err != nil {
+ return
+ }
+
+ ret = &Context{Name: name, Content: string(content)}
+ return
+}
+
+type Context struct {
+ Name string
+ Content string
+
+ contexts *Contexts
+}
+
+// Save the session on disk
+func (o *Context) Save() (err error) {
+ err = o.contexts.Save(o.Name, []byte(o.Content))
+ return err
+}
diff --git a/db/db.go b/db/db.go
new file mode 100644
index 0000000..3a593f9
--- /dev/null
+++ b/db/db.go
@@ -0,0 +1,87 @@
+package db
+
+import (
+ "fmt"
+ "github.com/joho/godotenv"
+ "os"
+ "path/filepath"
+ "time"
+)
+
+func NewDb(dir string) (db *Db) {
+
+ db = &Db{Dir: dir}
+
+ db.EnvFilePath = db.FilePath(".env")
+
+ db.Patterns = &Patterns{
+ Storage: &Storage{Label: "Patterns", Dir: db.FilePath("patterns"), ItemIsDir: true},
+ SystemPatternFile: "system.md",
+ UniquePatternsFilePath: db.FilePath("unique_patterns.txt"),
+ }
+ db.Sessions = &Sessions{&Storage{Label: "Sessions", Dir: db.FilePath("sessions")}}
+ db.Contexts = &Contexts{&Storage{Label: "Contexts", Dir: db.FilePath("contexts")}}
+
+ return
+}
+
+type Db struct {
+ Dir string
+
+ Patterns *Patterns
+ Sessions *Sessions
+ Contexts *Contexts
+
+ EnvFilePath string
+}
+
+func (o *Db) Configure() (err error) {
+ if err = os.MkdirAll(o.Dir, os.ModePerm); err != nil {
+ return
+ }
+
+ if err = o.LoadEnvFile(); err != nil {
+ return
+ }
+
+ if err = o.Patterns.Configure(); err != nil {
+ return
+ }
+
+ if err = o.Sessions.Configure(); err != nil {
+ return
+ }
+
+ if err = o.Contexts.Configure(); err != nil {
+ return
+ }
+
+ return
+}
+
+func (o *Db) LoadEnvFile() (err error) {
+ if err = godotenv.Load(o.EnvFilePath); err != nil {
+ err = fmt.Errorf("error loading .env file: %s", err)
+ }
+ return
+}
+
+func (o *Db) IsEnvFileExists() (ret bool) {
+ _, err := os.Stat(o.EnvFilePath)
+ ret = !os.IsNotExist(err)
+ return
+}
+
+func (o *Db) SaveEnv(content string) (err error) {
+ err = os.WriteFile(o.EnvFilePath, []byte(content), 0644)
+ return
+}
+
+func (o *Db) FilePath(fileName string) (ret string) {
+ return filepath.Join(o.Dir, fileName)
+}
+
+type DirectoryChange struct {
+ Dir string
+ Timestamp time.Time
+}
diff --git a/db/patterns.go b/db/patterns.go
new file mode 100644
index 0000000..2dfb954
--- /dev/null
+++ b/db/patterns.go
@@ -0,0 +1,52 @@
+package db
+
+import (
+ "fmt"
+ "os"
+ "path/filepath"
+ "strings"
+)
+
+type Patterns struct {
+ *Storage
+ SystemPatternFile string
+ UniquePatternsFilePath string
+}
+
+// GetByName finds a pattern by name and returns the pattern as an entry or an error
+func (o *Patterns) GetByName(name string) (ret *Pattern, err error) {
+ patternPath := filepath.Join(o.Dir, name, o.SystemPatternFile)
+
+ var pattern []byte
+ if pattern, err = os.ReadFile(patternPath); err != nil {
+ return
+ }
+ ret = &Pattern{
+ Name: name,
+ Pattern: string(pattern),
+ }
+ return
+}
+
+func (o *Patterns) LatestPatterns(latestNumber int) (err error) {
+ var contents []byte
+ if contents, err = os.ReadFile(o.UniquePatternsFilePath); err != nil {
+ err = fmt.Errorf("could not read unique patterns file. Pleas run --updatepatterns (%s)", err)
+ return
+ }
+ uniquePatterns := strings.Split(string(contents), "\n")
+ if latestNumber > len(uniquePatterns) {
+ latestNumber = len(uniquePatterns)
+ }
+
+ for i := len(uniquePatterns) - 1; i > len(uniquePatterns)-latestNumber-1; i-- {
+ fmt.Println(uniquePatterns[i])
+ }
+ return
+}
+
+type Pattern struct {
+ Name string
+ Description string
+ Pattern string
+}
diff --git a/db/sessions.go b/db/sessions.go
new file mode 100644
index 0000000..54c3500
--- /dev/null
+++ b/db/sessions.go
@@ -0,0 +1,68 @@
+package db
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+
+ "github.com/danielmiessler/fabric/common"
+)
+
+type Sessions struct {
+ *Storage
+}
+
+func (o *Sessions) LoadOrCreateSession(name string) (ret *Session, err error) {
+ if name == "" {
+ return &Session{}, nil
+ }
+
+ path := o.BuildFilePath(name)
+ if _, statErr := os.Stat(path); errors.Is(statErr, os.ErrNotExist) {
+ fmt.Printf("Creating new session: %s\n", name)
+ ret = &Session{Name: name, sessions: o}
+ } else {
+ ret, err = o.loadSession(name)
+ }
+ return
+}
+
+// LoadSession Load a session from file
+func (o *Sessions) LoadSession(name string) (ret *Session, err error) {
+ if name == "" {
+ return &Session{}, nil
+ }
+ ret, err = o.loadSession(name)
+ return
+}
+
+func (o *Sessions) loadSession(name string) (ret *Session, err error) {
+ ret = &Session{Name: name, sessions: o}
+ if err = o.LoadAsJson(name, &ret.Messages); err != nil {
+ return
+ }
+ return
+}
+
+type Session struct {
+ Name string
+ Messages []*common.Message
+
+ sessions *Sessions
+}
+
+func (o *Session) Append(messages ...*common.Message) {
+ o.Messages = append(o.Messages, messages...)
+}
+
+// Save the session on disk
+func (o *Session) Save() (err error) {
+ var jsonBytes []byte
+ if jsonBytes, err = json.Marshal(o.Messages); err == nil {
+ err = o.sessions.Save(o.Name, jsonBytes)
+ } else {
+ err = fmt.Errorf("could not marshal session %o: %o", o.Name, err)
+ }
+ return
+}
diff --git a/db/storage.go b/db/storage.go
new file mode 100644
index 0000000..611745f
--- /dev/null
+++ b/db/storage.go
@@ -0,0 +1,138 @@
+package db
+
+import (
+ "encoding/json"
+ "fmt"
+ "github.com/samber/lo"
+ "os"
+ "path/filepath"
+)
+
+type Storage struct {
+ Label string
+ Dir string
+ ItemIsDir bool
+ ItemExtension string
+}
+
+func (o *Storage) Configure() (err error) {
+ if err = os.MkdirAll(o.Dir, os.ModePerm); err != nil {
+ return
+ }
+ return
+}
+
+// GetNames finds all patterns in the patterns directory and enters the id, name, and pattern into a slice of Entry structs. it returns these entries or an error
+func (o *Storage) GetNames() (ret []string, err error) {
+ var entries []os.DirEntry
+ if entries, err = os.ReadDir(o.Dir); err != nil {
+ err = fmt.Errorf("could not read items from directory: %v", err)
+ return
+ }
+
+ if o.ItemIsDir {
+ ret = lo.FilterMap(entries, func(item os.DirEntry, index int) (ret string, ok bool) {
+ if ok = item.IsDir(); ok {
+ ret = item.Name()
+ }
+ return
+ })
+ } else {
+ ret = lo.FilterMap(entries, func(item os.DirEntry, index int) (ret string, ok bool) {
+ if ok = !item.IsDir() && filepath.Ext(item.Name()) == o.ItemExtension; ok {
+ ret = item.Name()
+ }
+ return
+ })
+ }
+ return
+}
+
+func (o *Storage) ListNames() (err error) {
+ var names []string
+ if names, err = o.GetNames(); err != nil {
+ return
+ }
+
+ if len(names) == 0 {
+ fmt.Printf("\nNo %v\n", o.Label)
+ return
+ }
+
+ fmt.Printf("\n%v:\n", o.Label)
+ for _, item := range names {
+ fmt.Printf("\t%s\n", item)
+ }
+ return
+}
+
+func (o *Storage) BuildFilePathByName(name string) (ret string) {
+ ret = o.BuildFilePath(o.buildFileName(name))
+ return
+}
+
+func (o *Storage) BuildFilePath(fileName string) (ret string) {
+ ret = filepath.Join(o.Dir, fileName)
+ return
+}
+
+func (o *Storage) buildFileName(name string) string {
+ return fmt.Sprintf("%s%v", name, o.ItemExtension)
+}
+
+func (o *Storage) Delete(name string) (err error) {
+ if err = os.Remove(o.BuildFilePathByName(name)); err != nil {
+ err = fmt.Errorf("could not delete %s: %v", name, err)
+ }
+ return
+}
+
+func (o *Storage) Exists(name string) (ret bool) {
+ _, err := os.Stat(o.BuildFilePathByName(name))
+ ret = !os.IsNotExist(err)
+ return
+}
+
+func (o *Storage) Rename(oldName, newName string) (err error) {
+ if err = os.Rename(o.BuildFilePathByName(oldName), o.BuildFilePathByName(newName)); err != nil {
+ err = fmt.Errorf("could not rename %s to %s: %v", oldName, newName, err)
+ }
+ return
+}
+
+func (o *Storage) Save(name string, content []byte) (err error) {
+ if err = os.WriteFile(o.BuildFilePathByName(name), content, 0644); err != nil {
+ err = fmt.Errorf("could not save %s: %v", name, err)
+ }
+ return
+}
+
+func (o *Storage) Load(name string) (ret []byte, err error) {
+ if ret, err = os.ReadFile(o.BuildFilePathByName(name)); err != nil {
+ err = fmt.Errorf("could not load %s: %v", name, err)
+ }
+ return
+}
+
+func (o *Storage) SaveAsJson(name string, item interface{}) (err error) {
+ var jsonString []byte
+ if jsonString, err = json.Marshal(item); err == nil {
+ err = o.Save(name, jsonString)
+ } else {
+ err = fmt.Errorf("could not marshal %s: %s", name, err)
+ }
+
+ return err
+}
+
+func (o *Storage) LoadAsJson(name string, item interface{}) (err error) {
+ var content []byte
+ if content, err = o.Load(name); err != nil {
+ return
+ }
+
+ if err = json.Unmarshal(content, &item); err != nil {
+ err = fmt.Errorf("could not unmarshal %s: %s", name, err)
+ }
+ return
+}
diff --git a/github-contributing.py b/github-contributing.py
deleted file mode 100644
index 1d26c60..0000000
--- a/github-contributing.py
+++ /dev/null
@@ -1,82 +0,0 @@
-import sys
-import argparse
-import subprocess
-
-def get_github_username():
- """Retrieve GitHub username from local Git configuration."""
- result = subprocess.run(['git', 'config', '--get', 'user.name'], capture_output=True, text=True)
- if result.returncode == 0 and result.stdout:
- return result.stdout.strip()
- else:
- raise Exception("Failed to retrieve GitHub username from Git config.")
-
-def update_fork():
- # Sync your fork's main branch with the original repository's main branch
- print("Updating fork...")
- subprocess.run(['git', 'fetch', 'upstream'], check=True) # Fetch the branches and their respective commits from the upstream repository
- subprocess.run(['git', 'checkout', 'main'], check=True) # Switch to your local main branch
- subprocess.run(['git', 'merge', 'upstream/main'], check=True) # Merge changes from upstream/main into your local main branch
- subprocess.run(['git', 'push', 'origin', 'main'], check=True) # Push the updated main branch to your fork on GitHub
- print("Fork updated successfully.")
-
-def create_branch(branch_name):
- print(f"Creating new branch '{branch_name}'...")
- subprocess.run(['git', 'checkout', '-b', branch_name], check=True)
- print(f"Branch '{branch_name}' created and switched to.")
-
-def push_changes(branch_name, commit_message):
- # Push your local changes to your fork on GitHub
- print("Pushing changes to fork...")
- subprocess.run(['git', 'checkout', branch_name], check=True) # Switch to the branch where your changes are
- subprocess.run(['git', 'add', '.'], check=True) # Stage all changes for commit
- subprocess.run(['git', 'commit', '-m', commit_message], check=True) # Commit the staged changes with a custom message
- subprocess.run(['git', 'push', 'fork', branch_name], check=True) # Push the commit to the same branch in your fork
- print("Changes pushed successfully.")
-
-def create_pull_request(branch_name, pr_title, pr_file):
- # Create a pull request on GitHub using the GitHub CLI
- print("Creating pull request...")
- github_username = get_github_username()
- with open(pr_file, 'r') as file:
- pr_body = file.read() # Read the PR description from a markdown file
- subprocess.run(['gh', 'pr', 'create',
- '--base', 'main',
- '--head', f'{github_username}:{branch_name}',
- '--title', pr_title,
- '--body', pr_body], check=True) # Create a pull request with the specified title and markdown body
- print("Pull request created successfully.")
-
-def main():
- parser = argparse.ArgumentParser(description="Automate your GitHub workflow")
- subparsers = parser.add_subparsers(dest='command', help='Available commands')
-
- # Subparser for updating fork
- parser_update = subparsers.add_parser('update-fork', help="Update fork with the latest from the original repository")
-
- parser_create_branch = subparsers.add_parser('create-branch', help="Create a new branch")
- parser_create_branch.add_argument('--branch-name', required=True, help="The name for the new branch")
-
- # Subparser for pushing changes
- parser_push = subparsers.add_parser('push-changes', help="Push local changes to the fork")
- parser_push.add_argument('--branch-name', required=True, help="The name of the branch you are working on")
- parser_push.add_argument('--commit-message', required=True, help="The commit message for your changes")
-
- # Subparser for creating a pull request
- parser_pr = subparsers.add_parser('create-pr', help="Create a pull request to the original repository")
- parser_pr.add_argument('--branch-name', required=True, help="The name of the branch the pull request is from")
- parser_pr.add_argument('--pr-title', required=True, help="The title of your pull request")
- parser_pr.add_argument('--pr-file', required=True, help="The markdown file path for your pull request description")
-
- args = parser.parse_args()
-
- if args.command == 'update-fork':
- update_fork()
- elif args.command == 'create-branch':
- create_branch(args.branch_name)
- elif args.command == 'push-changes':
- push_changes(args.branch_name, args.commit_message)
- elif args.command == 'create-pr':
- create_pull_request(args.branch_name, args.pr_title, args.pr_file)
-
-if __name__ == '__main__':
- main()
diff --git a/go.mod b/go.mod
new file mode 100644
index 0000000..086de6e
--- /dev/null
+++ b/go.mod
@@ -0,0 +1,72 @@
+module github.com/danielmiessler/fabric
+
+go 1.22.5
+
+toolchain go1.22.6
+
+require (
+ github.com/atotto/clipboard v0.1.4
+ github.com/go-git/go-git/v5 v5.12.0
+ github.com/google/generative-ai-go v0.17.0
+ github.com/jessevdk/go-flags v1.6.1
+ github.com/joho/godotenv v1.5.1
+ github.com/liushuangls/go-anthropic/v2 v2.6.0
+ github.com/ollama/ollama v0.3.6
+ github.com/otiai10/copy v1.14.0
+ github.com/pkg/errors v0.9.1
+ github.com/samber/lo v1.47.0
+ github.com/sashabaranov/go-openai v1.28.2
+ google.golang.org/api v0.192.0
+ gopkg.in/gookit/color.v1 v1.1.6
+)
+
+require (
+ cloud.google.com/go v0.115.0 // indirect
+ cloud.google.com/go/ai v0.8.0 // indirect
+ cloud.google.com/go/auth v0.8.1 // indirect
+ cloud.google.com/go/auth/oauth2adapt v0.2.3 // indirect
+ cloud.google.com/go/compute/metadata v0.5.0 // indirect
+ cloud.google.com/go/longrunning v0.5.7 // indirect
+ dario.cat/mergo v1.0.0 // indirect
+ github.com/Microsoft/go-winio v0.6.1 // indirect
+ github.com/ProtonMail/go-crypto v1.0.0 // indirect
+ github.com/cloudflare/circl v1.3.7 // indirect
+ github.com/cyphar/filepath-securejoin v0.2.4 // indirect
+ github.com/emirpasic/gods v1.18.1 // indirect
+ github.com/felixge/httpsnoop v1.0.4 // indirect
+ github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
+ github.com/go-git/go-billy/v5 v5.5.0 // indirect
+ github.com/go-logr/logr v1.4.2 // indirect
+ github.com/go-logr/stdr v1.2.2 // indirect
+ github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+ github.com/google/s2a-go v0.1.8 // indirect
+ github.com/google/uuid v1.6.0 // indirect
+ github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
+ github.com/googleapis/gax-go/v2 v2.13.0 // indirect
+ github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+ github.com/kevinburke/ssh_config v1.2.0 // indirect
+ github.com/pjbgf/sha1cd v0.3.0 // indirect
+ github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
+ github.com/skeema/knownhosts v1.2.2 // indirect
+ github.com/xanzy/ssh-agent v0.3.3 // indirect
+ go.opencensus.io v0.24.0 // indirect
+ go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0 // indirect
+ go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0 // indirect
+ go.opentelemetry.io/otel v1.26.0 // indirect
+ go.opentelemetry.io/otel/metric v1.26.0 // indirect
+ go.opentelemetry.io/otel/trace v1.26.0 // indirect
+ golang.org/x/crypto v0.25.0 // indirect
+ golang.org/x/mod v0.17.0 // indirect
+ golang.org/x/net v0.27.0 // indirect
+ golang.org/x/oauth2 v0.22.0 // indirect
+ golang.org/x/sync v0.8.0 // indirect
+ golang.org/x/sys v0.24.0 // indirect
+ golang.org/x/text v0.16.0 // indirect
+ golang.org/x/time v0.6.0 // indirect
+ golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
+ google.golang.org/genproto/googleapis/api v0.0.0-20240711142825-46eb208f015d // indirect
+ google.golang.org/genproto/googleapis/rpc v0.0.0-20240730163845-b1a4ccb954bf // indirect
+ google.golang.org/grpc v1.64.1 // indirect
+ google.golang.org/protobuf v1.34.2 // indirect
+ gopkg.in/warnings.v0 v0.1.2 // indirect
+)
diff --git a/go.sum b/go.sum
new file mode 100644
index 0000000..94b3785
--- /dev/null
+++ b/go.sum
@@ -0,0 +1,300 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.115.0 h1:CnFSK6Xo3lDYRoBKEcAtia6VSC837/ZkJuRduSFnr14=
+cloud.google.com/go v0.115.0/go.mod h1:8jIM5vVgoAEoiVxQ/O4BFTfHqulPZgs/ufEzMcFMdWU=
+cloud.google.com/go/ai v0.8.0 h1:rXUEz8Wp2OlrM8r1bfmpF2+VKqc1VJpafE3HgzRnD/w=
+cloud.google.com/go/ai v0.8.0/go.mod h1:t3Dfk4cM61sytiggo2UyGsDVW3RF1qGZaUKDrZFyqkE=
+cloud.google.com/go/auth v0.8.1 h1:QZW9FjC5lZzN864p13YxvAtGUlQ+KgRL+8Sg45Z6vxo=
+cloud.google.com/go/auth v0.8.1/go.mod h1:qGVp/Y3kDRSDZ5gFD/XPUfYQ9xW1iI7q8RIRoCyBbJc=
+cloud.google.com/go/auth/oauth2adapt v0.2.3 h1:MlxF+Pd3OmSudg/b1yZ5lJwoXCEaeedAguodky1PcKI=
+cloud.google.com/go/auth/oauth2adapt v0.2.3/go.mod h1:tMQXOfZzFuNuUxOypHlQEXgdfX5cuhwU+ffUuXRJE8I=
+cloud.google.com/go/compute/metadata v0.5.0 h1:Zr0eK8JbFv6+Wi4ilXAR8FJ3wyNdpxHKJNPos6LTZOY=
+cloud.google.com/go/compute/metadata v0.5.0/go.mod h1:aHnloV2TPI38yx4s9+wAZhHykWvVCfu7hQbF+9CWoiY=
+cloud.google.com/go/longrunning v0.5.7 h1:WLbHekDbjK1fVFD3ibpFFVoyizlLRl73I7YKuAKilhU=
+cloud.google.com/go/longrunning v0.5.7/go.mod h1:8GClkudohy1Fxm3owmBGid8W0pSgodEMwEAztp38Xng=
+dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
+dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
+github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow=
+github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
+github.com/ProtonMail/go-crypto v1.0.0 h1:LRuvITjQWX+WIfr930YHG2HNfjR1uOfyf5vE0kC2U78=
+github.com/ProtonMail/go-crypto v1.0.0/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
+github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
+github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
+github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
+github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
+github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
+github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
+github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
+github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
+github.com/cloudflare/circl v1.3.7 h1:qlCDlTPz2n9fu58M0Nh1J/JzcFpfgkFHHX3O35r5vcU=
+github.com/cloudflare/circl v1.3.7/go.mod h1:sRTcRWXGLrKw6yIGJ+l7amYJFfAXbZG0kBSc8r4zxgA=
+github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
+github.com/cyphar/filepath-securejoin v0.2.4 h1:Ugdm7cg7i6ZK6x3xDF1oEu1nfkyfH53EtKeQYTC3kyg=
+github.com/cyphar/filepath-securejoin v0.2.4/go.mod h1:aPGpWjXOXUn2NCNjFvBE6aRxGGx79pTxQpKOJNYHHl4=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/elazarl/goproxy v0.0.0-20230808193330-2592e75ae04a h1:mATvB/9r/3gvcejNsXKSkQ6lcIaNec2nyfOdlTBR2lU=
+github.com/elazarl/goproxy v0.0.0-20230808193330-2592e75ae04a/go.mod h1:Ro8st/ElPeALwNFlcTpWmkr6IoMFfkjXAvTHpevnDsM=
+github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
+github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
+github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
+github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
+github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
+github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/gliderlabs/ssh v0.3.7 h1:iV3Bqi942d9huXnzEF2Mt+CY9gLu8DNM4Obd+8bODRE=
+github.com/gliderlabs/ssh v0.3.7/go.mod h1:zpHEXBstFnQYtGnB8k8kQLol82umzn/2/snG7alWVD8=
+github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
+github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
+github.com/go-git/go-billy/v5 v5.5.0 h1:yEY4yhzCDuMGSv83oGxiBotRzhwhNr8VZyphhiu+mTU=
+github.com/go-git/go-billy/v5 v5.5.0/go.mod h1:hmexnoNsr2SJU1Ju67OaNz5ASJY3+sHgFRpCtpDCKow=
+github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
+github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
+github.com/go-git/go-git/v5 v5.12.0 h1:7Md+ndsjrzZxbddRDZjF14qK+NN56sy6wkqaVrjZtys=
+github.com/go-git/go-git/v5 v5.12.0/go.mod h1:FTM9VKtnI2m65hNI/TenDDDnUf2Q9FHnXYjuz9i5OEY=
+github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
+github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
+github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
+github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
+github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
+github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
+github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
+github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
+github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
+github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
+github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
+github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
+github.com/google/generative-ai-go v0.17.0 h1:kUmCXUIwJouD7I7ev3OmxzzQVICyhIWAxaXk2yblCMY=
+github.com/google/generative-ai-go v0.17.0/go.mod h1:JYolL13VG7j79kM5BtHz4qwONHkeJQzOCkKXnpqtS/E=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
+github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
+github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs=
+github.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0=
+github.com/googleapis/gax-go/v2 v2.13.0 h1:yitjD5f7jQHhyDsnhKEBU52NdvvdSeGzlAnDPT0hH1s=
+github.com/googleapis/gax-go/v2 v2.13.0/go.mod h1:Z/fvTZXF8/uw7Xu5GuslPw+bplx6SS338j1Is2S+B7A=
+github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
+github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
+github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bBK4=
+github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPGBnNLMooyc=
+github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
+github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
+github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
+github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/liushuangls/go-anthropic/v2 v2.6.0 h1:hkgLQPD04wL4lFrV5ZoGlIyy4f6P+brIuRlzn2S8K9s=
+github.com/liushuangls/go-anthropic/v2 v2.6.0/go.mod h1:8BKv/fkeTaL5R9R9bGkaknYBueyw2WxY20o7bImbOek=
+github.com/ollama/ollama v0.3.6 h1:nA/N0AmjP327po5cZDGLqI40nl+aeei0pD0dLa92ypE=
+github.com/ollama/ollama v0.3.6/go.mod h1:YrWoNkFnPOYsnDvsf/Ztb1wxU9/IXrNsQHqcxbY2r94=
+github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
+github.com/onsi/gomega v1.27.10/go.mod h1:RsS8tutOdbdgzbPtzzATp12yT7kM5I5aElG3evPbQ0M=
+github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
+github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
+github.com/otiai10/mint v1.5.1 h1:XaPLeE+9vGbuyEHem1JNk3bYc7KKqyI/na0/mLd/Kks=
+github.com/otiai10/mint v1.5.1/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM=
+github.com/pjbgf/sha1cd v0.3.0 h1:4D5XXmUUBUl/xQ6IjCkEAbqXskkq/4O7LmGn0AqMDs4=
+github.com/pjbgf/sha1cd v0.3.0/go.mod h1:nZ1rrWOcGJ5uZgEEVL1VUM9iRQiZvWdbZjkKyFzPPsI=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
+github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
+github.com/samber/lo v1.47.0 h1:z7RynLwP5nbyRscyvcD043DWYoOcYRv3mV8lBeqOCLc=
+github.com/samber/lo v1.47.0/go.mod h1:RmDH9Ct32Qy3gduHQuKJ3gW1fMHAnE/fAzQuf6He5cU=
+github.com/sashabaranov/go-openai v1.28.2 h1:Q3pi34SuNYNN7YrqpHlHbpeYlf75ljgHOAVM/r1yun0=
+github.com/sashabaranov/go-openai v1.28.2/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
+github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
+github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
+github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
+github.com/skeema/knownhosts v1.2.2 h1:Iug2P4fLmDw9f41PB6thxUkNUkJzB5i+1/exaj40L3A=
+github.com/skeema/knownhosts v1.2.2/go.mod h1:xYbVRSPxqBZFrdmDyMmsOs+uX1UZC3nTN3ThzgDxUwo=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
+github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
+go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0 h1:A3SayB3rNyt+1S6qpI9mHPkeHTZbD7XILEqWnYZb2l0=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.51.0/go.mod h1:27iA5uvhuRNmalO+iEUdVn5ZMj2qy10Mm+XRIpRmyuU=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0 h1:Xs2Ncz0gNihqu9iosIZ5SkBbWo5T8JhhLJFMQL1qmLI=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0/go.mod h1:vy+2G/6NvVMpwGX/NyLqcC41fxepnuKHk16E6IZUcJc=
+go.opentelemetry.io/otel v1.26.0 h1:LQwgL5s/1W7YiiRwxf03QGnWLb2HW4pLiAhaA5cZXBs=
+go.opentelemetry.io/otel v1.26.0/go.mod h1:UmLkJHUAidDval2EICqBMbnAd0/m2vmpf/dAM+fvFs4=
+go.opentelemetry.io/otel/metric v1.26.0 h1:7S39CLuY5Jgg9CrnA9HHiEjGMF/X2VHvoXGgSllRz30=
+go.opentelemetry.io/otel/metric v1.26.0/go.mod h1:SY+rHOI4cEawI9a7N1A4nIg/nTQXe1ccCNWYOJUrpX4=
+go.opentelemetry.io/otel/trace v1.26.0 h1:1ieeAUb4y0TE26jUFrCIXKpTuVK7uJGN9/Z/2LP5sQA=
+go.opentelemetry.io/otel/trace v1.26.0/go.mod h1:4iDxvGDQuUkHve82hJJ8UqrwswHYsZuWCBllGV2U2y0=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
+golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
+golang.org/x/crypto v0.25.0 h1:ypSNr+bnYL2YhwoMt2zPxHFmbAN1KZs/njMG3hxUp30=
+golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
+golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
+golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
+golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
+golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys=
+golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.22.0 h1:BzDx2FehcG7jJwgWLELCdmLuxk2i+x9UDpSiss2u0ZA=
+golang.org/x/oauth2 v0.22.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
+golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.24.0 h1:Twjiwq9dn6R1fQcyiK+wQyHWfaz/BJB+YIpzU/Cv3Xg=
+golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
+golang.org/x/term v0.22.0 h1:BbsgPEJULsl2fV/AT3v15Mjva5yXKQDyKf+TbDz7QJk=
+golang.org/x/term v0.22.0/go.mod h1:F3qCibpT5AMpCRfhfT53vVJwhLtIVHhB9XDjfFvnMI4=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
+golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
+golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
+golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U=
+golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
+golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
+golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+google.golang.org/api v0.192.0 h1:PljqpNAfZaaSpS+TnANfnNAXKdzHM/B9bKhwRlo7JP0=
+google.golang.org/api v0.192.0/go.mod h1:9VcphjvAxPKLmSxVSzPlSRXy/5ARMEw5bf58WoVXafQ=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
+google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
+google.golang.org/genproto/googleapis/api v0.0.0-20240711142825-46eb208f015d h1:kHjw/5UfflP/L5EbledDrcG4C2597RtymmGRZvHiCuY=
+google.golang.org/genproto/googleapis/api v0.0.0-20240711142825-46eb208f015d/go.mod h1:mw8MG/Qz5wfgYr6VqVCiZcHe/GJEfI+oGGDCohaVgB0=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240730163845-b1a4ccb954bf h1:liao9UHurZLtiEwBgT9LMOnKYsHze6eA6w1KQCMVN2Q=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240730163845-b1a4ccb954bf/go.mod h1:Ue6ibwXGpU+dqIcODieyLOcgj7z8+IcskoNIgZxtrFY=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
+google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
+google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
+google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
+google.golang.org/grpc v1.64.1 h1:LKtvyfbX3UGVPFcGqJ9ItpVWW6oN/2XqTxfAnwRRXiA=
+google.golang.org/grpc v1.64.1/go.mod h1:hiQF4LFZelK2WKaP6W0L92zGHtiQdZxk8CrSdvyjeP0=
+google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
+google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
+google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
+google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
+google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
+google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
+google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
+google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
+google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+gopkg.in/gookit/color.v1 v1.1.6 h1:5fB10p6AUFjhd2ayq9JgmJWr9WlTrguFdw3qlYtKNHk=
+gopkg.in/gookit/color.v1 v1.1.6/go.mod h1:IcEkFGaveVShJ+j8ew+jwe9epHyGpJ9IrptHmW3laVY=
+gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
+gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
diff --git a/images/fabric-logo-gif.gif b/images/fabric-logo-gif.gif
deleted file mode 100644
index cf87b03..0000000
Binary files a/images/fabric-logo-gif.gif and /dev/null differ
diff --git a/installer/__init__.py b/installer/__init__.py
deleted file mode 100644
index 6206887..0000000
--- a/installer/__init__.py
+++ /dev/null
@@ -1,5 +0,0 @@
-from .client.cli import main as cli, main_save, main_ts, main_yt
-from .server import (
- run_api_server,
- run_webui_server,
-)
diff --git a/installer/client/cli/README.md b/installer/client/cli/README.md
deleted file mode 100644
index eb0b15b..0000000
--- a/installer/client/cli/README.md
+++ /dev/null
@@ -1,3 +0,0 @@
-# The `fabric` client
-
-Please see the main project's README.md for the latest documentation.
diff --git a/installer/client/cli/__init__.py b/installer/client/cli/__init__.py
deleted file mode 100644
index 3395da7..0000000
--- a/installer/client/cli/__init__.py
+++ /dev/null
@@ -1,4 +0,0 @@
-from .fabric import main
-from .yt import main as main_yt
-from .ts import main as main_ts
-from .save import cli as main_save
diff --git a/installer/client/cli/agents/trip_planner/main.py b/installer/client/cli/agents/trip_planner/main.py
deleted file mode 100644
index cf92341..0000000
--- a/installer/client/cli/agents/trip_planner/main.py
+++ /dev/null
@@ -1,89 +0,0 @@
-from crewai import Crew
-from textwrap import dedent
-from .trip_agents import TripAgents
-from .trip_tasks import TripTasks
-import os
-from dotenv import load_dotenv
-
-current_directory = os.path.dirname(os.path.realpath(__file__))
-config_directory = os.path.expanduser("~/.config/fabric")
-env_file = os.path.join(config_directory, ".env")
-load_dotenv(env_file)
-os.environ['OPENAI_MODEL_NAME'] = 'gpt-4-0125-preview'
-
-
-class TripCrew:
-
- def __init__(self, origin, cities, date_range, interests):
- self.cities = cities
- self.origin = origin
- self.interests = interests
- self.date_range = date_range
-
- def run(self):
- agents = TripAgents()
- tasks = TripTasks()
-
- city_selector_agent = agents.city_selection_agent()
- local_expert_agent = agents.local_expert()
- travel_concierge_agent = agents.travel_concierge()
-
- identify_task = tasks.identify_task(
- city_selector_agent,
- self.origin,
- self.cities,
- self.interests,
- self.date_range
- )
- gather_task = tasks.gather_task(
- local_expert_agent,
- self.origin,
- self.interests,
- self.date_range
- )
- plan_task = tasks.plan_task(
- travel_concierge_agent,
- self.origin,
- self.interests,
- self.date_range
- )
-
- crew = Crew(
- agents=[
- city_selector_agent, local_expert_agent, travel_concierge_agent
- ],
- tasks=[identify_task, gather_task, plan_task],
- verbose=True
- )
-
- result = crew.kickoff()
- return result
-
-
-class planner_cli:
- def ask(self):
- print("## Welcome to Trip Planner Crew")
- print('-------------------------------')
- location = input(
- dedent("""
- From where will you be traveling from?
- """))
- cities = input(
- dedent("""
- What are the cities options you are interested in visiting?
- """))
- date_range = input(
- dedent("""
- What is the date range you are interested in traveling?
- """))
- interests = input(
- dedent("""
- What are some of your high level interests and hobbies?
- """))
-
- trip_crew = TripCrew(location, cities, date_range, interests)
- result = trip_crew.run()
- print("\n\n########################")
- print("## Here is you Trip Plan")
- print("########################\n")
- print(result)
diff --git a/installer/client/cli/agents/trip_planner/tools/__init__.py b/installer/client/cli/agents/trip_planner/tools/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/installer/client/cli/agents/trip_planner/tools/browser_tools.py b/installer/client/cli/agents/trip_planner/tools/browser_tools.py
deleted file mode 100644
index 36a10a9..0000000
--- a/installer/client/cli/agents/trip_planner/tools/browser_tools.py
+++ /dev/null
@@ -1,38 +0,0 @@
-import json
-import os
-
-import requests
-from crewai import Agent, Task
-from langchain.tools import tool
-from unstructured.partition.html import partition_html
-
-
-class BrowserTools():
-
- @tool("Scrape website content")
- def scrape_and_summarize_website(website):
- """Useful to scrape and summarize a website content"""
- url = f"https://chrome.browserless.io/content?token={os.environ['BROWSERLESS_API_KEY']}"
- payload = json.dumps({"url": website})
- headers = {'cache-control': 'no-cache', 'content-type': 'application/json'}
- response = requests.request("POST", url, headers=headers, data=payload)
- elements = partition_html(text=response.text)
- content = "\n\n".join([str(el) for el in elements])
- content = [content[i:i + 8000] for i in range(0, len(content), 8000)]
- summaries = []
- for chunk in content:
- agent = Agent(
- role='Principal Researcher',
- goal=
- 'Do amazing researches and summaries based on the content you are working with',
- backstory=
- "You're a Principal Researcher at a big company and you need to do a research about a given topic.",
- allow_delegation=False)
- task = Task(
- agent=agent,
- description=
- f'Analyze and summarize the content bellow, make sure to include the most relevant information in the summary, return only the summary nothing else.\n\nCONTENT\n----------\n{chunk}'
- )
- summary = task.execute()
- summaries.append(summary)
- return "\n\n".join(summaries)
diff --git a/installer/client/cli/agents/trip_planner/tools/calculator_tools.py b/installer/client/cli/agents/trip_planner/tools/calculator_tools.py
deleted file mode 100644
index 51f0066..0000000
--- a/installer/client/cli/agents/trip_planner/tools/calculator_tools.py
+++ /dev/null
@@ -1,15 +0,0 @@
-from langchain.tools import tool
-
-class CalculatorTools():
-
- @tool("Make a calculation")
- def calculate(operation):
- """Useful to perform any mathematical calculations,
- like sum, minus, multiplication, division, etc.
- The input to this tool should be a mathematical
- expression, a couple examples are `200*7` or `5000/2*10`
- """
- try:
- return eval(operation)
- except SyntaxError:
- return "Error: Invalid syntax in mathematical expression"
diff --git a/installer/client/cli/agents/trip_planner/tools/search_tools.py b/installer/client/cli/agents/trip_planner/tools/search_tools.py
deleted file mode 100644
index 5c1d0ae..0000000
--- a/installer/client/cli/agents/trip_planner/tools/search_tools.py
+++ /dev/null
@@ -1,37 +0,0 @@
-import json
-import os
-
-import requests
-from langchain.tools import tool
-
-
-class SearchTools():
-
- @tool("Search the internet")
- def search_internet(query):
- """Useful to search the internet
- about a a given topic and return relevant results"""
- top_result_to_return = 4
- url = "https://google.serper.dev/search"
- payload = json.dumps({"q": query})
- headers = {
- 'X-API-KEY': os.environ['SERPER_API_KEY'],
- 'content-type': 'application/json'
- }
- response = requests.request("POST", url, headers=headers, data=payload)
- # check if there is an organic key
- if 'organic' not in response.json():
- return "Sorry, I couldn't find anything about that, there could be an error with you serper api key."
- else:
- results = response.json()['organic']
- string = []
- for result in results[:top_result_to_return]:
- try:
- string.append('\n'.join([
- f"Title: {result['title']}", f"Link: {result['link']}",
- f"Snippet: {result['snippet']}", "\n-----------------"
- ]))
- except KeyError:
- next
-
- return '\n'.join(string)
diff --git a/installer/client/cli/agents/trip_planner/trip_agents.py b/installer/client/cli/agents/trip_planner/trip_agents.py
deleted file mode 100644
index 0ca9e2f..0000000
--- a/installer/client/cli/agents/trip_planner/trip_agents.py
+++ /dev/null
@@ -1,45 +0,0 @@
-from crewai import Agent
-
-from .tools.browser_tools import BrowserTools
-from .tools.calculator_tools import CalculatorTools
-from .tools.search_tools import SearchTools
-
-
-class TripAgents():
-
- def city_selection_agent(self):
- return Agent(
- role='City Selection Expert',
- goal='Select the best city based on weather, season, and prices',
- backstory='An expert in analyzing travel data to pick ideal destinations',
- tools=[
- SearchTools.search_internet,
- BrowserTools.scrape_and_summarize_website,
- ],
- verbose=True)
-
- def local_expert(self):
- return Agent(
- role='Local Expert at this city',
- goal='Provide the BEST insights about the selected city',
- backstory="""A knowledgeable local guide with extensive information
- about the city, it's attractions and customs""",
- tools=[
- SearchTools.search_internet,
- BrowserTools.scrape_and_summarize_website,
- ],
- verbose=True)
-
- def travel_concierge(self):
- return Agent(
- role='Amazing Travel Concierge',
- goal="""Create the most amazing travel itineraries with budget and
- packing suggestions for the city""",
- backstory="""Specialist in travel planning and logistics with
- decades of experience""",
- tools=[
- SearchTools.search_internet,
- BrowserTools.scrape_and_summarize_website,
- CalculatorTools.calculate,
- ],
- verbose=True)
diff --git a/installer/client/cli/agents/trip_planner/trip_tasks.py b/installer/client/cli/agents/trip_planner/trip_tasks.py
deleted file mode 100644
index 8b3181e..0000000
--- a/installer/client/cli/agents/trip_planner/trip_tasks.py
+++ /dev/null
@@ -1,83 +0,0 @@
-from crewai import Task
-from textwrap import dedent
-from datetime import date
-
-
-class TripTasks():
-
- def identify_task(self, agent, origin, cities, interests, range):
- return Task(description=dedent(f"""
- Analyze and select the best city for the trip based
- on specific criteria such as weather patterns, seasonal
- events, and travel costs. This task involves comparing
- multiple cities, considering factors like current weather
- conditions, upcoming cultural or seasonal events, and
- overall travel expenses.
-
- Your final answer must be a detailed
- report on the chosen city, and everything you found out
- about it, including the actual flight costs, weather
- forecast and attractions.
- {self.__tip_section()}
-
- Traveling from: {origin}
- City Options: {cities}
- Trip Date: {range}
- Traveler Interests: {interests}
- """),
- agent=agent)
-
- def gather_task(self, agent, origin, interests, range):
- return Task(description=dedent(f"""
- As a local expert on this city you must compile an
- in-depth guide for someone traveling there and wanting
- to have THE BEST trip ever!
- Gather information about key attractions, local customs,
- special events, and daily activity recommendations.
- Find the best spots to go to, the kind of place only a
- local would know.
- This guide should provide a thorough overview of what
- the city has to offer, including hidden gems, cultural
- hotspots, must-visit landmarks, weather forecasts, and
- high level costs.
-
- The final answer must be a comprehensive city guide,
- rich in cultural insights and practical tips,
- tailored to enhance the travel experience.
- {self.__tip_section()}
-
- Trip Date: {range}
- Traveling from: {origin}
- Traveler Interests: {interests}
- """),
- agent=agent)
-
- def plan_task(self, agent, origin, interests, range):
- return Task(description=dedent(f"""
- Expand this guide into a a full 7-day travel
- itinerary with detailed per-day plans, including
- weather forecasts, places to eat, packing suggestions,
- and a budget breakdown.
-
- You MUST suggest actual places to visit, actual hotels
- to stay and actual restaurants to go to.
-
- This itinerary should cover all aspects of the trip,
- from arrival to departure, integrating the city guide
- information with practical travel logistics.
-
- Your final answer MUST be a complete expanded travel plan,
- formatted as markdown, encompassing a daily schedule,
- anticipated weather conditions, recommended clothing and
- items to pack, and a detailed budget, ensuring THE BEST
- TRIP EVER, Be specific and give it a reason why you picked
- # up each place, what make them special! {self.__tip_section()}
-
- Trip Date: {range}
- Traveling from: {origin}
- Traveler Interests: {interests}
- """),
- agent=agent)
-
- def __tip_section(self):
- return "If you do your BEST WORK, I'll tip you $100!"
\ No newline at end of file
diff --git a/installer/client/cli/fabric.py b/installer/client/cli/fabric.py
deleted file mode 100755
index 05dc3cc..0000000
--- a/installer/client/cli/fabric.py
+++ /dev/null
@@ -1,212 +0,0 @@
-from .utils import Standalone, Update, Setup, Alias, run_electron_app
-import argparse
-import sys
-import os
-
-
-script_directory = os.path.dirname(os.path.realpath(__file__))
-
-
-def main():
- parser = argparse.ArgumentParser(
- description="An open source framework for augmenting humans using AI."
- )
- parser.add_argument("--text", "-t", help="Text to extract summary from")
- parser.add_argument(
- "--copy", "-C", help="Copy the response to the clipboard", action="store_true"
- )
- parser.add_argument(
- '--agents', '-a',
- help="Use praisonAI to create an AI agent and then use it. ex: 'write me a movie script'", action="store_true"
- )
-
- parser.add_argument(
- "--output",
- "-o",
- help="Save the response to a file",
- nargs="?",
- const="analyzepaper.txt",
- default=None,
- )
- parser.add_argument('--session', '-S',
- help="Continue your previous conversation. Default is your previous conversation", nargs="?", const="default")
- parser.add_argument(
- '--clearsession', help="deletes indicated session. Use 'all' to delete all sessions")
- parser.add_argument('--sessionlog', help="View the log of a session")
- parser.add_argument(
- '--listsessions', help="List all sessions", action="store_true")
- parser.add_argument(
- "--gui", help="Use the GUI (Node and npm need to be installed)", action="store_true")
- parser.add_argument(
- "--stream",
- "-s",
- help="Use this option if you want to see the results in realtime. NOTE: You will not be able to pipe the output into another command.",
- action="store_true",
- )
- parser.add_argument(
- "--list", "-l", help="List available patterns", action="store_true"
- )
- parser.add_argument(
- '--temp', help="set the temperature for the model. Default is 0", default=0, type=float)
- parser.add_argument(
- '--top_p', help="set the top_p for the model. Default is 1", default=1, type=float)
- parser.add_argument(
- '--frequency_penalty', help="set the frequency penalty for the model. Default is 0.1", default=0.1, type=float)
- parser.add_argument(
- '--presence_penalty', help="set the presence penalty for the model. Default is 0.1", default=0.1, type=float)
- parser.add_argument(
- "--update", "-u", help="Update patterns", action="store_true")
- parser.add_argument("--pattern", "-p", help="The pattern (prompt) to use")
- parser.add_argument(
- "--setup", help="Set up your fabric instance", action="store_true"
- )
- parser.add_argument('--changeDefaultModel',
- help="Change the default model. For a list of available models, use the --listmodels flag.")
-
- parser.add_argument(
- "--model", "-m", help="Select the model to use"
- )
- parser.add_argument(
- "--listmodels", help="List all available models", action="store_true"
- )
- parser.add_argument('--remoteOllamaServer',
- help='The URL of the remote ollamaserver to use. ONLY USE THIS if you are using a local ollama server in an non-default location or port')
- parser.add_argument('--context', '-c',
- help="Use Context file (context.md) to add context to your pattern", action="store_true")
-
- args = parser.parse_args()
- home_holder = os.path.expanduser("~")
- config = os.path.join(home_holder, ".config", "fabric")
- config_patterns_directory = os.path.join(config, "patterns")
- config_context = os.path.join(config, "context.md")
- env_file = os.path.join(config, ".env")
- if not os.path.exists(config):
- os.makedirs(config)
- if args.setup:
- Setup().run()
- Alias().execute()
- sys.exit()
- if not os.path.exists(env_file) or not os.path.exists(config_patterns_directory):
- print("Please run --setup to set up your API key and download patterns.")
- sys.exit()
- if not os.path.exists(config_patterns_directory):
- Update()
- Alias()
- sys.exit()
- if args.changeDefaultModel:
- Setup().default_model(args.changeDefaultModel)
- sys.exit()
- if args.gui:
- run_electron_app()
- sys.exit()
- if args.update:
- Update()
- Alias()
- sys.exit()
- if args.context:
- if not os.path.exists(os.path.join(config, "context.md")):
- print("Please create a context.md file in ~/.config/fabric")
- sys.exit()
- if args.agents:
- standalone = Standalone(args)
- text = "" # Initialize text variable
- # Check if an argument was provided to --agents
- if args.text:
- text = args.text
- else:
- text = standalone.get_cli_input()
- if text:
- standalone = Standalone(args)
- standalone.agents(text)
- sys.exit()
- if args.session:
- from .helper import Session
- session = Session()
- if args.session == "default":
- session_file = session.find_most_recent_file()
- if session_file is None:
- args.session = "default"
- else:
- args.session = session_file.split("/")[-1]
- if args.clearsession:
- from .helper import Session
- session = Session()
- session.clear_session(args.clearsession)
- if args.clearsession == "all":
- print(f"All sessions cleared")
- else:
- print(f"Session {args.clearsession} cleared")
- sys.exit()
- if args.sessionlog:
- from .helper import Session
- session = Session()
- print(session.session_log(args.sessionlog))
- sys.exit()
- if args.listsessions:
- from .helper import Session
- session = Session()
- session.list_sessions()
- sys.exit()
- standalone = Standalone(args, args.pattern)
- if args.list:
- try:
- direct = sorted(os.listdir(config_patterns_directory))
- for d in direct:
- print(d)
- sys.exit()
- except FileNotFoundError:
- print("No patterns found")
- sys.exit()
- if args.listmodels:
- gptmodels, localmodels, claudemodels, googlemodels = standalone.fetch_available_models()
- print("GPT Models:")
- for model in gptmodels:
- print(model)
- print("\nLocal Models:")
- for model in localmodels:
- print(model)
- print("\nClaude Models:")
- for model in claudemodels:
- print(model)
- print("\nGoogle Models:")
- for model in googlemodels:
- print(model)
- sys.exit()
- if args.text is not None:
- text = args.text
- else:
- text = standalone.get_cli_input()
- if args.stream and not args.context:
- if args.remoteOllamaServer:
- standalone.streamMessage(text, host=args.remoteOllamaServer)
- else:
- standalone.streamMessage(text)
- sys.exit()
- if args.stream and args.context:
- with open(config_context, "r") as f:
- context = f.read()
- if args.remoteOllamaServer:
- standalone.streamMessage(
- text, context=context, host=args.remoteOllamaServer)
- else:
- standalone.streamMessage(text, context=context)
- sys.exit()
- elif args.context:
- with open(config_context, "r") as f:
- context = f.read()
- if args.remoteOllamaServer:
- standalone.sendMessage(
- text, context=context, host=args.remoteOllamaServer)
- else:
- standalone.sendMessage(text, context=context)
- sys.exit()
- else:
- if args.remoteOllamaServer:
- standalone.sendMessage(text, host=args.remoteOllamaServer)
- else:
- standalone.sendMessage(text)
- sys.exit()
-
-
-if __name__ == "__main__":
- main()
diff --git a/installer/client/cli/helper.py b/installer/client/cli/helper.py
deleted file mode 100644
index 3f5db8a..0000000
--- a/installer/client/cli/helper.py
+++ /dev/null
@@ -1,75 +0,0 @@
-import os
-import sys
-
-
-class Session:
- def __init__(self):
- home_folder = os.path.expanduser("~")
- config_folder = os.path.join(home_folder, ".config", "fabric")
- self.sessions_folder = os.path.join(config_folder, "sessions")
- if not os.path.exists(self.sessions_folder):
- os.makedirs(self.sessions_folder)
-
- def find_most_recent_file(self):
- # Ensure the directory exists
- directory = self.sessions_folder
- if not os.path.exists(directory):
- print("Directory does not exist:", directory)
- return None
-
- # List all files in the directory
- full_path_files = [os.path.join(directory, file) for file in os.listdir(
- directory) if os.path.isfile(os.path.join(directory, file))]
-
- # If no files are found, return None
- if not full_path_files:
- return None
-
- # Find the file with the most recent modification time
- most_recent_file = max(full_path_files, key=os.path.getmtime)
-
- return most_recent_file
-
- def save_to_session(self, system, user, response, fileName):
- file = os.path.join(self.sessions_folder, fileName)
- with open(file, "a+") as f:
- f.write(f"{system}\n")
- f.write(f"{user}\n")
- f.write(f"{response}\n")
-
- def read_from_session(self, filename):
- file = os.path.join(self.sessions_folder, filename)
- if not os.path.exists(file):
- return None
- with open(file, "r") as f:
- return f.read()
-
- def clear_session(self, session):
- if session == "all":
- for file in os.listdir(self.sessions_folder):
- os.remove(os.path.join(self.sessions_folder, file))
- else:
- os.remove(os.path.join(self.sessions_folder, session))
-
- def session_log(self, session):
- file = os.path.join(self.sessions_folder, session)
- if not os.path.exists(file):
- return None
- with open(file, "r") as f:
- return f.read()
-
- def list_sessions(self):
- sessionlist = os.listdir(self.sessions_folder)
- find_most_recent_file_result = self.find_most_recent_file()
- if find_most_recent_file_result is not None:
- most_recent = find_most_recent_file_result.split("/")[-1]
- for session in sessionlist:
- with open(os.path.join(self.sessions_folder, session), "r") as f:
- firstline = f.readline().strip()
- secondline = f.readline().strip()
- if session == most_recent:
- print(f"{session} **default** \"{firstline}\n{secondline}\n\"")
- else:
- print(f"{session} \"{firstline}\n{secondline}\n\"")
- else:
- print('No files present in sessions directory')
diff --git a/installer/client/cli/save.py b/installer/client/cli/save.py
deleted file mode 100755
index e4f850c..0000000
--- a/installer/client/cli/save.py
+++ /dev/null
@@ -1,122 +0,0 @@
-import argparse
-import os
-import sys
-from datetime import datetime
-
-from dotenv import load_dotenv
-
-DEFAULT_CONFIG = "~/.config/fabric/.env"
-PATH_KEY = "FABRIC_OUTPUT_PATH"
-FM_KEY = "FABRIC_FRONTMATTER_TAGS"
-load_dotenv(os.path.expanduser(DEFAULT_CONFIG))
-DATE_FORMAT = os.getenv("SAVE_DATE_FORMAT", "%Y-%m-%d")
-
-def main(tag, tags, silent, fabric):
- out = os.getenv(PATH_KEY)
- if out is None:
- print(f"'{PATH_KEY}' not set in {DEFAULT_CONFIG} or in your environment.")
- sys.exit(1)
-
- out = os.path.expanduser(out)
-
- if not os.path.isdir(out):
- print(f"'{out}' does not exist. Create it and try again.")
- sys.exit(1)
-
- if not out.endswith("/"):
- out += "/"
-
- if len(sys.argv) < 2:
- print(f"'{sys.argv[0]}' takes a single argument to tag your summary")
- sys.exit(1)
-
- if DATE_FORMAT:
- yyyymmdd = datetime.now().strftime(DATE_FORMAT)
- target = f"{out}{yyyymmdd}-{tag}.md"
- else:
- target = f"{out}{tag}.md"
-
- # don't clobber existing files- add an incremented number to the end instead
- would_clobber = True
- inc = 0
- while would_clobber:
- if inc > 0:
- if DATE_FORMAT:
- target = f"{out}{yyyymmdd}-{tag}-{inc}.md"
- else:
- target = f"{out}{tag}-{inc}.md"
- if os.path.exists(target):
- inc += 1
- else:
- would_clobber = False
-
- # YAML frontmatter stubs for things like Obsidian
- # Prevent a NoneType ending up in the tags
- frontmatter_tags = os.getenv(FM_KEY) or "" if fabric else ""
- with open(target, "w") as fp:
- if frontmatter_tags or len(tags) != 0:
- fp.write("---\n")
- now = datetime.now().strftime("%Y-%m-%d %H:%M")
- fp.write(f"generation_date: {now}\n")
- fp.write(f"tags: {frontmatter_tags} {tag} {' '.join(tags)}\n")
- fp.write("---\n")
-
- # function like 'tee' and split the output to a file and STDOUT
- for line in sys.stdin:
- if not silent:
- print(line, end="")
- fp.write(line)
-
-
-def cli():
- parser = argparse.ArgumentParser(
- description=(
- 'save: a "tee-like" utility to pipeline saving of content, '
- "while keeping the output stream intact. Can optionally generate "
- '"frontmatter" for PKM utilities like Obsidian via the '
- '"FABRIC_FRONTMATTER" environment variable'
- )
- )
- parser.add_argument(
- "stub",
- nargs="?",
- help=(
- "stub to describe your content. Use quotes if you have spaces. "
- "Resulting format is YYYY-MM-DD-stub.md by default"
- ),
- )
- parser.add_argument(
- "-t,",
- "--tag",
- required=False,
- action="append",
- default=[],
- help=(
- "add an additional frontmatter tag. Use this argument multiple times"
- "for multiple tags"
- ),
- )
- parser.add_argument(
- "-n",
- "--nofabric",
- required=False,
- action="store_false",
- help="don't use the fabric tags, only use tags from --tag",
- )
- parser.add_argument(
- "-s",
- "--silent",
- required=False,
- action="store_true",
- help="don't use STDOUT for output, only save to the file",
- )
- args = parser.parse_args()
-
- if args.stub:
- main(args.stub, args.tag, args.silent, args.nofabric)
- else:
- parser.print_help()
-
-
-if __name__ == "__main__":
- cli()
diff --git a/installer/client/cli/ts.py b/installer/client/cli/ts.py
deleted file mode 100644
index fa98ba2..0000000
--- a/installer/client/cli/ts.py
+++ /dev/null
@@ -1,110 +0,0 @@
-from dotenv import load_dotenv
-from pydub import AudioSegment
-from openai import OpenAI
-import os
-import argparse
-
-
-class Whisper:
- def __init__(self):
- env_file = os.path.expanduser("~/.config/fabric/.env")
- load_dotenv(env_file)
- try:
- apikey = os.environ["OPENAI_API_KEY"]
- self.client = OpenAI()
- self.client.api_key = apikey
- except KeyError:
- print("OPENAI_API_KEY not found in environment variables.")
-
- except FileNotFoundError:
- print("No API key found. Use the --apikey option to set the key")
- self.whole_response = []
-
- def split_audio(self, file_path):
- """
- Splits the audio file into segments of the given length.
-
- Args:
- - file_path: The path to the audio file.
- - segment_length_ms: Length of each segment in milliseconds.
-
- Returns:
- - A list of audio segments.
- """
- audio = AudioSegment.from_file(file_path)
- segments = []
- segment_length_ms = 10 * 60 * 1000 # 10 minutes in milliseconds
- for start_ms in range(0, len(audio), segment_length_ms):
- end_ms = start_ms + segment_length_ms
- segment = audio[start_ms:end_ms]
- segments.append(segment)
-
- return segments
-
- def process_segment(self, segment):
- """ Transcribe an audio file and print the transcript.
-
- Args:
- audio_file (str): The path to the audio file to be transcribed.
-
- Returns:
- None
- """
-
- try:
- # if audio_file.startswith("http"):
- # response = requests.get(audio_file)
- # response.raise_for_status()
- # with tempfile.NamedTemporaryFile(delete=False) as f:
- # f.write(response.content)
- # audio_file = f.name
- audio_file = open(segment, "rb")
- response = self.client.audio.transcriptions.create(
- model="whisper-1",
- file=audio_file
- )
- self.whole_response.append(response.text)
-
- except Exception as e:
- print(f"Error: {e}")
-
- def process_file(self, audio_file):
- """ Transcribe an audio file and print the transcript.
-
- Args:
- audio_file (str): The path to the audio file to be transcribed.
-
- Returns:
- None
- """
-
- try:
- # if audio_file.startswith("http"):
- # response = requests.get(audio_file)
- # response.raise_for_status()
- # with tempfile.NamedTemporaryFile(delete=False) as f:
- # f.write(response.content)
- # audio_file = f.name
-
- segments = self.split_audio(audio_file)
- for i, segment in enumerate(segments):
- segment_file_path = f"segment_{i}.mp3"
- segment.export(segment_file_path, format="mp3")
- self.process_segment(segment_file_path)
- print(' '.join(self.whole_response))
-
- except Exception as e:
- print(f"Error: {e}")
-
-
-def main():
- parser = argparse.ArgumentParser(description="Transcribe an audio file.")
- parser.add_argument(
- "audio_file", help="The path to the audio file to be transcribed.")
- args = parser.parse_args()
- whisper = Whisper()
- whisper.process_file(args.audio_file)
-
-
-if __name__ == "__main__":
- main()
diff --git a/installer/client/cli/utils.py b/installer/client/cli/utils.py
deleted file mode 100644
index e8715f4..0000000
--- a/installer/client/cli/utils.py
+++ /dev/null
@@ -1,897 +0,0 @@
-import requests
-import os
-from openai import OpenAI, APIConnectionError
-import asyncio
-import pyperclip
-import sys
-import platform
-from dotenv import load_dotenv
-import zipfile
-import tempfile
-import subprocess
-import shutil
-from youtube_transcript_api import YouTubeTranscriptApi
-
-current_directory = os.path.dirname(os.path.realpath(__file__))
-config_directory = os.path.expanduser("~/.config/fabric")
-env_file = os.path.join(config_directory, ".env")
-
-
-class Standalone:
- def __init__(self, args, pattern="", env_file="~/.config/fabric/.env"):
- """ Initialize the class with the provided arguments and environment file.
-
- Args:
- args: The arguments for initialization.
- pattern: The pattern to be used (default is an empty string).
- env_file: The path to the environment file (default is "~/.config/fabric/.env").
-
- Returns:
- None
-
- Raises:
- KeyError: If the "OPENAI_API_KEY" is not found in the environment variables.
- FileNotFoundError: If no API key is found in the environment variables.
- """
-
- # Expand the tilde to the full path
- if args is None:
- args = type('Args', (), {})()
- env_file = os.path.expanduser(env_file)
- self.client = None
- load_dotenv(env_file)
- if "OPENAI_API_KEY" in os.environ:
- api_key = os.environ['OPENAI_API_KEY']
- self.client = OpenAI(api_key=api_key)
- self.local = False
- self.config_pattern_directory = config_directory
- self.pattern = pattern
- self.args = args
- self.model = getattr(args, 'model', None)
- if not self.model:
- self.model = os.environ.get('DEFAULT_MODEL', None)
- if not self.model:
- self.model = 'gpt-4-turbo-preview'
- self.claude = False
- sorted_gpt_models, ollamaList, claudeList, googleList = self.fetch_available_models()
- self.sorted_gpt_models = sorted_gpt_models
- self.ollamaList = ollamaList
- self.claudeList = claudeList
- self.googleList = googleList
- self.local = self.model in ollamaList
- self.claude = self.model in claudeList
- self.google = self.model in googleList
-
- async def localChat(self, messages, host=''):
- from ollama import AsyncClient
- response = None
- if host:
- response = await AsyncClient(host=host).chat(model=self.model, messages=messages)
- else:
- response = await AsyncClient().chat(model=self.model, messages=messages)
- print(response['message']['content'])
- copy = self.args.copy
- if copy:
- pyperclip.copy(response['message']['content'])
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(response['message']['content'])
-
- async def localStream(self, messages, host=''):
- from ollama import AsyncClient
- buffer = ""
- if host:
- async for part in await AsyncClient(host=host).chat(model=self.model, messages=messages, stream=True):
- buffer += part['message']['content']
- print(part['message']['content'], end='', flush=True)
- else:
- async for part in await AsyncClient().chat(model=self.model, messages=messages, stream=True):
- buffer += part['message']['content']
- print(part['message']['content'], end='', flush=True)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(buffer)
- if self.args.copy:
- pyperclip.copy(buffer)
-
- async def claudeStream(self, system, user):
- from anthropic import AsyncAnthropic
- self.claudeApiKey = os.environ["CLAUDE_API_KEY"]
- Streamingclient = AsyncAnthropic(api_key=self.claudeApiKey)
- buffer = ""
- async with Streamingclient.messages.stream(
- max_tokens=4096,
- system=system,
- messages=[user],
- model=self.model, temperature=self.args.temp, top_p=self.args.top_p
- ) as stream:
- async for text in stream.text_stream:
- buffer += text
- print(text, end="", flush=True)
- print()
- if self.args.copy:
- pyperclip.copy(buffer)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(buffer)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, buffer, self.args.session)
- message = await stream.get_final_message()
-
- async def claudeChat(self, system, user, copy=False):
- from anthropic import Anthropic
- self.claudeApiKey = os.environ["CLAUDE_API_KEY"]
- client = Anthropic(api_key=self.claudeApiKey)
- message = None
- message = client.messages.create(
- max_tokens=4096,
- system=system,
- messages=[user],
- model=self.model,
- temperature=self.args.temp, top_p=self.args.top_p
- )
- print(message.content[0].text)
- copy = self.args.copy
- if copy:
- pyperclip.copy(message.content[0].text)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(message.content[0].text)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, message.content[0].text, self.args.session)
-
- async def googleChat(self, system, user, copy=False):
- import google.generativeai as genai
- self.googleApiKey = os.environ["GOOGLE_API_KEY"]
- genai.configure(api_key=self.googleApiKey)
- model = genai.GenerativeModel(
- model_name=self.model, system_instruction=system)
- response = model.generate_content(user)
- print(response.text)
- if copy:
- pyperclip.copy(response.text)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(response.text)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, response.text, self.args.session)
-
- async def googleStream(self, system, user, copy=False):
- import google.generativeai as genai
- buffer = ""
- self.googleApiKey = os.environ["GOOGLE_API_KEY"]
- genai.configure(api_key=self.googleApiKey)
- model = genai.GenerativeModel(
- model_name=self.model, system_instruction=system)
- response = model.generate_content(user, stream=True)
- for chunk in response:
- buffer += chunk.text
- print(chunk.text)
- if copy:
- pyperclip.copy(buffer)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(buffer)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, buffer, self.args.session)
-
- def streamMessage(self, input_data: str, context="", host=''):
- """ Stream a message and handle exceptions.
-
- Args:
- input_data (str): The input data for the message.
-
- Returns:
- None: If the pattern is not found.
-
- Raises:
- FileNotFoundError: If the pattern file is not found.
- """
-
- wisdomFilePath = os.path.join(
- config_directory, f"patterns/{self.pattern}/system.md"
- )
- session_message = ""
- user = ""
- if self.args.session:
- from .helper import Session
- session = Session()
- session_message = session.read_from_session(
- self.args.session)
- if session_message:
- user = session_message + '\n' + input_data
- else:
- user = input_data
- user_message = {"role": "user", "content": f"{input_data}"}
- wisdom_File = wisdomFilePath
- buffer = ""
- system = ""
- if self.pattern:
- try:
- with open(wisdom_File, "r") as f:
- if context:
- system = context + '\n\n' + f.read()
- if session_message:
- system = session_message + '\n' + system
- else:
- system = f.read()
- if session_message:
- system = session_message + '\n' + system
- system_message = {"role": "system", "content": system}
- messages = [system_message, user_message]
- except FileNotFoundError:
- print("pattern not found")
- return
- else:
- if session_message:
- user_message['content'] = session_message + \
- '\n' + user_message['content']
- if context:
- messages = [
- {"role": "system", "content": context}, user_message]
- else:
- messages = [user_message]
- try:
- if self.local:
- if host:
- asyncio.run(self.localStream(messages, host=host))
- else:
- asyncio.run(self.localStream(messages))
- elif self.claude:
- from anthropic import AsyncAnthropic
- asyncio.run(self.claudeStream(system, user_message))
- elif self.google:
- if system == "":
- system = " "
- asyncio.run(self.googleStream(system, user_message['content']))
- else:
- stream = self.client.chat.completions.create(
- model=self.model,
- messages=messages,
- temperature=self.args.temp,
- top_p=self.args.top_p,
- frequency_penalty=self.args.frequency_penalty,
- presence_penalty=self.args.presence_penalty,
- stream=True,
- )
- for chunk in stream:
- if chunk.choices[0].delta.content is not None:
- char = chunk.choices[0].delta.content
- buffer += char
- if char not in ["\n", " "]:
- print(char, end="")
- elif char == " ":
- print(" ", end="") # Explicitly handle spaces
- elif char == "\n":
- print() # Handle newlines
- sys.stdout.flush()
- except Exception as e:
- if "All connection attempts failed" in str(e):
- print(
- "Error: cannot connect to llama2. If you have not already, please visit https://ollama.com for installation instructions")
- if "CLAUDE_API_KEY" in str(e):
- print(
- "Error: CLAUDE_API_KEY not found in environment variables. Please run --setup and add the key")
- if "overloaded_error" in str(e):
- print(
- "Error: Fabric is working fine, but claude is overloaded. Please try again later.")
- else:
- print(f"Error: {e}")
- print(e)
- if self.args.copy:
- pyperclip.copy(buffer)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(buffer)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, buffer, self.args.session)
-
- def sendMessage(self, input_data: str, context="", host=''):
- """ Send a message using the input data and generate a response.
-
- Args:
- input_data (str): The input data to be sent as a message.
-
- Returns:
- None
-
- Raises:
- FileNotFoundError: If the specified pattern file is not found.
- """
-
- wisdomFilePath = os.path.join(
- config_directory, f"patterns/{self.pattern}/system.md"
- )
- user = input_data
- user_message = {"role": "user", "content": f"{input_data}"}
- wisdom_File = os.path.join(current_directory, wisdomFilePath)
- system = ""
- session_message = ""
- if self.args.session:
- from .helper import Session
- session = Session()
- session_message = session.read_from_session(
- self.args.session)
- if self.pattern:
- try:
- with open(wisdom_File, "r") as f:
- if context:
- if session_message:
- system = session_message + '\n' + context + '\n\n' + f.read()
- else:
- system = context + '\n\n' + f.read()
- else:
- if session_message:
- system = session_message + '\n' + f.read()
- else:
- system = f.read()
- system_message = {"role": "system", "content": system}
- messages = [system_message, user_message]
- except FileNotFoundError:
- print("pattern not found")
- return
- else:
- if session_message:
- user_message['content'] = session_message + \
- '\n' + user_message['content']
- if context:
- messages = [
- {'role': 'system', 'content': context}, user_message]
- else:
- messages = [user_message]
- try:
- if self.local:
- if host:
- asyncio.run(self.localChat(messages, host=host))
- else:
- asyncio.run(self.localChat(messages))
- elif self.claude:
- asyncio.run(self.claudeChat(system, user_message))
- elif self.google:
- if system == "":
- system = " "
- asyncio.run(self.googleChat(system, user_message['content']))
- else:
- response = self.client.chat.completions.create(
- model=self.model,
- messages=messages,
- temperature=self.args.temp,
- top_p=self.args.top_p,
- frequency_penalty=self.args.frequency_penalty,
- presence_penalty=self.args.presence_penalty,
- )
- print(response.choices[0].message.content)
- if self.args.copy:
- pyperclip.copy(response.choices[0].message.content)
- if self.args.output:
- with open(self.args.output, "w") as f:
- f.write(response.choices[0].message.content)
- if self.args.session:
- from .helper import Session
- session = Session()
- session.save_to_session(
- system, user, response.choices[0], self.args.session)
- except Exception as e:
- if "All connection attempts failed" in str(e):
- print(
- "Error: cannot connect to llama2. If you have not already, please visit https://ollama.com for installation instructions")
- if "CLAUDE_API_KEY" in str(e):
- print(
- "Error: CLAUDE_API_KEY not found in environment variables. Please run --setup and add the key")
- if "overloaded_error" in str(e):
- print(
- "Error: Fabric is working fine, but claude is overloaded. Please try again later.")
- if "Attempted to call a sync iterator on an async stream" in str(e):
- print("Error: There is a problem connecting fabric with your local ollama installation. Please visit https://ollama.com for installation instructions. It is possible that you have chosen the wrong model. Please run fabric --listmodels to see the available models and choose the right one with fabric --model or fabric --changeDefaultModel. If this does not work. Restart your computer (always a good idea) and try again. If you are still having problems, please visit https://ollama.com for installation instructions.")
- else:
- print(f"Error: {e}")
- print(e)
-
- def fetch_available_models(self):
- gptlist = []
- fullOllamaList = []
- googleList = []
- if "CLAUDE_API_KEY" in os.environ:
- claudeList = ['claude-3-5-sonnet-20240620','claude-3-opus-20240229', 'claude-3-sonnet-20240229',
- 'claude-3-haiku-20240307', 'claude-2.1']
- else:
- claudeList = []
-
- try:
- if self.client:
- models = [model.id.strip()
- for model in self.client.models.list().data]
- if "/" in models[0] or "\\" in models[0]:
- gptlist = [item[item.rfind(
- "/") + 1:] if "/" in item else item[item.rfind("\\") + 1:] for item in models]
- else:
- gptlist = [item.strip()
- for item in models if item.startswith("gpt")]
- gptlist.sort()
- except APIConnectionError as e:
- pass
- except Exception as e:
- print(f"Error: {getattr(e.__context__, 'args', [''])[0]}")
- sys.exit()
-
- import ollama
- try:
- remoteOllamaServer = getattr(self.args, 'remoteOllamaServer', None)
- if remoteOllamaServer:
- client = ollama.Client(host=self.args.remoteOllamaServer)
- default_modelollamaList = client.list()['models']
- else:
- default_modelollamaList = ollama.list()['models']
- for model in default_modelollamaList:
- fullOllamaList.append(model['name'])
- except:
- fullOllamaList = []
- try:
- import google.generativeai as genai
- genai.configure(api_key=os.environ["GOOGLE_API_KEY"])
- for m in genai.list_models():
- if 'generateContent' in m.supported_generation_methods:
- googleList.append(m.name)
- except:
- googleList = []
-
- return gptlist, fullOllamaList, claudeList, googleList
-
- def get_cli_input(self):
- """ aided by ChatGPT; uses platform library
- accepts either piped input or console input
- from either Windows or Linux
-
- Args:
- none
- Returns:
- string from either user or pipe
- """
- system = platform.system()
- if system == 'Windows':
- if not sys.stdin.isatty(): # Check if input is being piped
- return sys.stdin.read().strip() # Read piped input
- else:
- # Prompt user for input from console
- return input("Enter Question: ")
- else:
- return sys.stdin.read()
-
- def agents(self, userInput):
- from praisonai import PraisonAI
- model = self.model
- os.environ["OPENAI_MODEL_NAME"] = model
- if model in self.sorted_gpt_models:
- os.environ["OPENAI_API_BASE"] = "https://api.openai.com/v1/"
- elif model in self.ollamaList:
- os.environ["OPENAI_API_BASE"] = "http://localhost:11434/v1"
- os.environ["OPENAI_API_KEY"] = "NA"
-
- elif model in self.claudeList:
- print("Claude is not supported in this mode")
- sys.exit()
- print("Starting PraisonAI...")
- praison_ai = PraisonAI(auto=userInput, framework="autogen")
- praison_ai.main()
-
-
-class Update:
- def __init__(self):
- """Initialize the object with default values."""
- self.repo_zip_url = "https://github.com/danielmiessler/fabric/archive/refs/heads/main.zip"
- self.config_directory = os.path.expanduser("~/.config/fabric")
- self.pattern_directory = os.path.join(
- self.config_directory, "patterns")
- os.makedirs(self.pattern_directory, exist_ok=True)
- print("Updating patterns...")
- self.update_patterns() # Start the update process immediately
-
- def update_patterns(self):
- """Update the patterns by downloading the zip from GitHub and extracting it."""
- with tempfile.TemporaryDirectory() as temp_dir:
- zip_path = os.path.join(temp_dir, "repo.zip")
- self.download_zip(self.repo_zip_url, zip_path)
- extracted_folder_path = self.extract_zip(zip_path, temp_dir)
- # The patterns folder will be inside "fabric-main" after extraction
- patterns_source_path = os.path.join(
- extracted_folder_path, "fabric-main", "patterns")
- if os.path.exists(patterns_source_path):
- # If the patterns directory already exists, remove it before copying over the new one
- if os.path.exists(self.pattern_directory):
- old_pattern_contents = os.listdir(self.pattern_directory)
- new_pattern_contents = os.listdir(patterns_source_path)
- custom_patterns = []
- for pattern in old_pattern_contents:
- if pattern not in new_pattern_contents:
- custom_patterns.append(pattern)
- if custom_patterns:
- for pattern in custom_patterns:
- custom_path = os.path.join(
- self.pattern_directory, pattern)
- shutil.move(custom_path, patterns_source_path)
- shutil.rmtree(self.pattern_directory)
- shutil.copytree(patterns_source_path, self.pattern_directory)
- print("Patterns updated successfully.")
- else:
- print("Patterns folder not found in the downloaded zip.")
-
- def download_zip(self, url, save_path):
- """Download the zip file from the specified URL."""
- response = requests.get(url)
- response.raise_for_status() # Check if the download was successful
- with open(save_path, 'wb') as f:
- f.write(response.content)
- print("Downloaded zip file successfully.")
-
- def extract_zip(self, zip_path, extract_to):
- """Extract the zip file to the specified directory."""
- with zipfile.ZipFile(zip_path, 'r') as zip_ref:
- zip_ref.extractall(extract_to)
- print("Extracted zip file successfully.")
- return extract_to # Return the path to the extracted contents
-
-
-class Alias:
- def __init__(self):
- self.config_files = []
- self.home_directory = os.path.expanduser("~")
- patternsFolder = os.path.join(
- self.home_directory, ".config/fabric/patterns")
- self.patterns = os.listdir(patternsFolder)
-
- def execute(self):
- with open(os.path.join(self.home_directory, ".config/fabric/fabric-bootstrap.inc"), "w") as w:
- for pattern in self.patterns:
- w.write(f"alias {pattern}='fabric --pattern {pattern}'\n")
-
-
-class Setup:
- def __init__(self):
- """ Initialize the object.
-
- Raises:
- OSError: If there is an error in creating the pattern directory.
- """
-
- self.config_directory = os.path.expanduser("~/.config/fabric")
- self.pattern_directory = os.path.join(
- self.config_directory, "patterns")
- os.makedirs(self.pattern_directory, exist_ok=True)
- self.shconfigs = []
- home = os.path.expanduser("~")
- if os.path.exists(os.path.join(home, ".bashrc")):
- self.shconfigs.append(os.path.join(home, ".bashrc"))
- if os.path.exists(os.path.join(home, ".bash_profile")):
- self.shconfigs.append(os.path.join(home, ".bash_profile"))
- if os.path.exists(os.path.join(home, ".zshrc")):
- self.shconfigs.append(os.path.join(home, ".zshrc"))
- self.env_file = os.path.join(self.config_directory, ".env")
- self.gptlist = []
- self.fullOllamaList = []
- self.googleList = []
- self.claudeList = ['claude-3-opus-20240229']
- load_dotenv(self.env_file)
- try:
- openaiapikey = os.environ["OPENAI_API_KEY"]
- self.openaiapi_key = openaiapikey
- except:
- pass
-
- def __ensure_env_file_created(self):
- """ Ensure that the environment file is created.
-
- Returns:
- None
-
- Raises:
- OSError: If the environment file cannot be created.
- """
- print("Creating empty environment file...")
- if not os.path.exists(self.env_file):
- with open(self.env_file, "w") as f:
- f.write("#No API key set\n")
- print("Environment file created.")
-
- def update_shconfigs(self):
- bootstrap_file = os.path.join(
- self.config_directory, "fabric-bootstrap.inc")
- sourceLine = f'if [ -f "{bootstrap_file}" ]; then . "{bootstrap_file}"; fi'
- for config in self.shconfigs:
- lines = None
- with open(config, 'r') as f:
- lines = f.readlines()
- with open(config, 'w') as f:
- for line in lines:
- if sourceLine not in line:
- f.write(line)
- f.write(sourceLine)
-
- def api_key(self, api_key):
- """ Set the OpenAI API key in the environment file.
-
- Args:
- api_key (str): The API key to be set.
-
- Returns:
- None
-
- Raises:
- OSError: If the environment file does not exist or cannot be accessed.
- """
- api_key = api_key.strip()
- if not os.path.exists(self.env_file) and api_key:
- with open(self.env_file, "w") as f:
- f.write(f"OPENAI_API_KEY={api_key}\n")
- print(f"OpenAI API key set to {api_key}")
- elif api_key:
- # erase the line OPENAI_API_KEY=key and write the new key
- with open(self.env_file, "r") as f:
- lines = f.readlines()
- with open(self.env_file, "w") as f:
- for line in lines:
- if "OPENAI_API_KEY" not in line:
- f.write(line)
- f.write(f"OPENAI_API_KEY={api_key}\n")
-
- def claude_key(self, claude_key):
- """ Set the Claude API key in the environment file.
-
- Args:
- claude_key (str): The API key to be set.
-
- Returns:
- None
-
- Raises:
- OSError: If the environment file does not exist or cannot be accessed.
- """
- claude_key = claude_key.strip()
- if os.path.exists(self.env_file) and claude_key:
- with open(self.env_file, "r") as f:
- lines = f.readlines()
- with open(self.env_file, "w") as f:
- for line in lines:
- if "CLAUDE_API_KEY" not in line:
- f.write(line)
- f.write(f"CLAUDE_API_KEY={claude_key}\n")
- elif claude_key:
- with open(self.env_file, "w") as f:
- f.write(f"CLAUDE_API_KEY={claude_key}\n")
-
- def google_key(self, google_key):
- """ Set the Google API key in the environment file.
-
- Args:
- google_key (str): The API key to be set.
-
- Returns:
- None
-
- Raises:
- OSError: If the environment file does not exist or cannot be accessed.
- """
- google_key = google_key.strip()
- if os.path.exists(self.env_file) and google_key:
- with open(self.env_file, "r") as f:
- lines = f.readlines()
- with open(self.env_file, "w") as f:
- for line in lines:
- if "GOOGLE_API_KEY" not in line:
- f.write(line)
- f.write(f"GOOGLE_API_KEY={google_key}\n")
- elif google_key:
- with open(self.env_file, "w") as f:
- f.write(f"GOOGLE_API_KEY={google_key}\n")
-
- def youtube_key(self, youtube_key):
- """ Set the YouTube API key in the environment file.
-
- Args:
- youtube_key (str): The API key to be set.
-
- Returns:
- None
-
- Raises:
- OSError: If the environment file does not exist or cannot be accessed.
- """
- youtube_key = youtube_key.strip()
- if os.path.exists(self.env_file) and youtube_key:
- with open(self.env_file, "r") as f:
- lines = f.readlines()
- with open(self.env_file, "w") as f:
- for line in lines:
- if "YOUTUBE_API_KEY" not in line:
- f.write(line)
- f.write(f"YOUTUBE_API_KEY={youtube_key}\n")
- elif youtube_key:
- with open(self.env_file, "w") as f:
- f.write(f"YOUTUBE_API_KEY={youtube_key}\n")
-
- def default_model(self, model):
- """Set the default model in the environment file.
-
- Args:
- model (str): The model to be set.
- """
- model = model.strip()
- env = os.path.expanduser("~/.config/fabric/.env")
- standalone = Standalone(args=[], pattern="")
- gpt, ollama, claude, google = standalone.fetch_available_models()
- allmodels = gpt + ollama + claude + google
- if model not in allmodels:
- print(
- f"Error: {model} is not a valid model. Please run fabric --listmodels to see the available models.")
- sys.exit()
-
- # Only proceed if the model is not empty
- if model:
- if os.path.exists(env):
- # Initialize a flag to track the presence of DEFAULT_MODEL
- there = False
- with open(env, "r") as f:
- lines = f.readlines()
-
- # Open the file again to write the changes
- with open(env, "w") as f:
- for line in lines:
- # Check each line to see if it contains DEFAULT_MODEL
- if "DEFAULT_MODEL=" in line:
- # Update the flag and the line with the new model
- there = True
- f.write(f'DEFAULT_MODEL={model}\n')
- else:
- # If the line does not contain DEFAULT_MODEL, write it unchanged
- f.write(line)
-
- # If DEFAULT_MODEL was not found in the file, add it
- if not there:
- f.write(f'DEFAULT_MODEL={model}\n')
-
- print(
- f"Default model changed to {model}. Please restart your terminal to use it.")
- else:
- print("No shell configuration file found.")
-
- def patterns(self):
- """ Method to update patterns and exit the system.
-
- Returns:
- None
- """
-
- Update()
-
- def run(self):
- """ Execute the Fabric program.
-
- This method prompts the user for their OpenAI API key, sets the API key in the Fabric object, and then calls the patterns method.
-
- Returns:
- None
- """
-
- print("Welcome to Fabric. Let's get started.")
- apikey = input(
- "Please enter your OpenAI API key. If you do not have one or if you have already entered it, press enter.\n")
- self.api_key(apikey)
- print("Please enter your claude API key. If you do not have one, or if you have already entered it, press enter.\n")
- claudekey = input()
- self.claude_key(claudekey)
- print("Please enter your Google API key. If you do not have one, or if you have already entered it, press enter.\n")
- googlekey = input()
- self.google_key(googlekey)
- print("Please enter your YouTube API key. If you do not have one, or if you have already entered it, press enter.\n")
- youtubekey = input()
- self.youtube_key(youtubekey)
- self.patterns()
- self.update_shconfigs()
- self.__ensure_env_file_created()
-
-
-class Transcribe:
- def youtube(video_id):
- """
- This method gets the transciption
- of a YouTube video designated with the video_id
-
- Input:
- the video id specifying a YouTube video
- an example url for a video: https://www.youtube.com/watch?v=vF-MQmVxnCs&t=306s
- the video id is vF-MQmVxnCs&t=306s
-
- Output:
- a transcript for the video
-
- Raises:
- an exception and prints error
-
-
- """
- try:
- transcript_list = YouTubeTranscriptApi.get_transcript(video_id)
- transcript = ""
- for segment in transcript_list:
- transcript += segment['text'] + " "
- return transcript.strip()
- except Exception as e:
- print("Error:", e)
- return None
-
-
-class AgentSetup:
- def apiKeys(self):
- """Method to set the API keys in the environment file.
-
- Returns:
- None
- """
-
- print("Welcome to Fabric. Let's get started.")
- browserless = input("Please enter your Browserless API key\n").strip()
- serper = input("Please enter your Serper API key\n").strip()
-
- # Entries to be added
- browserless_entry = f"BROWSERLESS_API_KEY={browserless}"
- serper_entry = f"SERPER_API_KEY={serper}"
-
- # Check and write to the file
- with open(env_file, "r+") as f:
- content = f.read()
-
- # Determine if the file ends with a newline
- if content.endswith('\n'):
- # If it ends with a newline, we directly write the new entries
- f.write(f"{browserless_entry}\n{serper_entry}\n")
- else:
- # If it does not end with a newline, add one before the new entries
- f.write(f"\n{browserless_entry}\n{serper_entry}\n")
-
-
-def run_electron_app():
- # Step 1: Set CWD to the directory of the script
- os.chdir(os.path.dirname(os.path.realpath(__file__)))
-
- # Step 2: Check for the './installer/client/gui' directory
- target_dir = '../gui'
- if not os.path.exists(target_dir):
- print(f"""The directory {
- target_dir} does not exist. Please check the path and try again.""")
- return
-
- # Step 3: Check for NPM installation
- try:
- subprocess.run(['npm', '--version'], check=True,
- stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
- except subprocess.CalledProcessError:
- print("NPM is not installed. Please install NPM and try again.")
- return
-
- # If this point is reached, NPM is installed.
- # Step 4: Change directory to the Electron app's directory
- os.chdir(target_dir)
-
- # Step 5: Run 'npm install' and 'npm start'
- try:
- print("Running 'npm install'... This might take a few minutes.")
- subprocess.run(['npm', 'install'], check=True)
- print(
- "'npm install' completed successfully. Starting the Electron app with 'npm start'...")
- subprocess.run(['npm', 'start'], check=True)
- except subprocess.CalledProcessError as e:
- print(f"An error occurred while executing NPM commands: {e}")
diff --git a/installer/client/cli/yt.py b/installer/client/cli/yt.py
deleted file mode 100644
index 4b02da4..0000000
--- a/installer/client/cli/yt.py
+++ /dev/null
@@ -1,151 +0,0 @@
-import re
-from googleapiclient.discovery import build
-from googleapiclient.errors import HttpError
-from youtube_transcript_api import YouTubeTranscriptApi
-from dotenv import load_dotenv
-from datetime import datetime
-import os
-import json
-import isodate
-import argparse
-import sys
-
-
-def get_video_id(url):
- # Extract video ID from URL
- pattern = r"(?:https?:\/\/)?(?:www\.)?(?:youtube\.com\/(?:[^\/\n\s]+\/\S+\/|(?:v|e(?:mbed)?)\/|\S*?[?&]v=)|youtu\.be\/)([a-zA-Z0-9_-]{11})"
- match = re.search(pattern, url)
- return match.group(1) if match else None
-
-
-def get_comments(youtube, video_id):
- comments = []
-
- try:
- # Fetch top-level comments
- request = youtube.commentThreads().list(
- part="snippet,replies",
- videoId=video_id,
- textFormat="plainText",
- maxResults=100 # Adjust based on needs
- )
-
- while request:
- response = request.execute()
- for item in response['items']:
- # Top-level comment
- topLevelComment = item['snippet']['topLevelComment']['snippet']['textDisplay']
- comments.append(topLevelComment)
-
- # Check if there are replies in the thread
- if 'replies' in item:
- for reply in item['replies']['comments']:
- replyText = reply['snippet']['textDisplay']
- # Add incremental spacing and a dash for replies
- comments.append(" - " + replyText)
-
- # Prepare the next page of comments, if available
- if 'nextPageToken' in response:
- request = youtube.commentThreads().list_next(
- previous_request=request, previous_response=response)
- else:
- request = None
-
- except HttpError as e:
- print(f"Failed to fetch comments: {e}")
-
- return comments
-
-
-
-def main_function(url, options):
- # Load environment variables from .env file
- load_dotenv(os.path.expanduser("~/.config/fabric/.env"))
-
- # Get YouTube API key from environment variable
- api_key = os.getenv("YOUTUBE_API_KEY")
- if not api_key:
- print("Error: YOUTUBE_API_KEY not found in ~/.config/fabric/.env")
- return
-
- # Extract video ID from URL
- video_id = get_video_id(url)
- if not video_id:
- print("Invalid YouTube URL")
- return
-
- try:
- # Initialize the YouTube API client
- youtube = build("youtube", "v3", developerKey=api_key)
-
- # Get video details
- video_response = youtube.videos().list(
- id=video_id, part="contentDetails,snippet").execute()
-
- # Extract video duration and convert to minutes
- duration_iso = video_response["items"][0]["contentDetails"]["duration"]
- duration_seconds = isodate.parse_duration(duration_iso).total_seconds()
- duration_minutes = round(duration_seconds / 60)
- # Set up metadata
- metadata = {}
- metadata['id'] = video_response['items'][0]['id']
- metadata['title'] = video_response['items'][0]['snippet']['title']
- metadata['channel'] = video_response['items'][0]['snippet']['channelTitle']
- metadata['published_at'] = video_response['items'][0]['snippet']['publishedAt']
-
- # Get video transcript
- try:
- transcript_list = YouTubeTranscriptApi.get_transcript(video_id, languages=[options.lang])
- transcript_text = " ".join([item["text"] for item in transcript_list])
- transcript_text = transcript_text.replace("\n", " ")
- except Exception as e:
- transcript_text = f"Transcript not available in the selected language ({options.lang}). ({e})"
-
- # Get comments if the flag is set
- comments = []
- if options.comments:
- comments = get_comments(youtube, video_id)
-
- # Output based on options
- if options.duration:
- print(duration_minutes)
- elif options.transcript:
- print(transcript_text.encode('utf-8').decode('unicode-escape'))
- elif options.comments:
- print(json.dumps(comments, indent=2))
- elif options.metadata:
- print(json.dumps(metadata, indent=2))
- else:
- # Create JSON object with all data
- output = {
- "transcript": transcript_text,
- "duration": duration_minutes,
- "comments": comments,
- "metadata": metadata
- }
- # Print JSON object
- print(json.dumps(output, indent=2))
- except HttpError as e:
- print(f"Error: Failed to access YouTube API. Please check your YOUTUBE_API_KEY and ensure it is valid: {e}")
-
-
-def main():
- parser = argparse.ArgumentParser(
- description='yt (video meta) extracts metadata about a video, such as the transcript, the video\'s duration, and now comments. By Daniel Miessler.')
- parser.add_argument('url', help='YouTube video URL')
- parser.add_argument('--duration', action='store_true', help='Output only the duration')
- parser.add_argument('--transcript', action='store_true', help='Output only the transcript')
- parser.add_argument('--comments', action='store_true', help='Output the comments on the video')
- parser.add_argument('--metadata', action='store_true', help='Output the video metadata')
- parser.add_argument('--lang', default='en', help='Language for the transcript (default: English)')
-
- args = parser.parse_args()
-
- if args.url is None:
- print("Error: No URL provided.")
- return
-
- main_function(args.url, args)
-
-if __name__ == "__main__":
- main()
diff --git a/installer/client/gui/.gitignore b/installer/client/gui/.gitignore
deleted file mode 100644
index d93463d..0000000
--- a/installer/client/gui/.gitignore
+++ /dev/null
@@ -1,3 +0,0 @@
-node_modules/
-dist/
-build/
diff --git a/installer/client/gui/README.md b/installer/client/gui/README.md
deleted file mode 100644
index 4bd2346..0000000
--- a/installer/client/gui/README.md
+++ /dev/null
@@ -1,21 +0,0 @@
-Fabric is not just a tool; it's a transformative step towards integrating the power of GPT prompts into your digital life. With Fabric, you have the ability to create a personal API that brings advanced GPT capabilities into various aspects of your digital environment. Whether you're looking to incorporate powerful GPT prompts into command line operations or extend their functionality to a wider network through a personal API, Fabric is designed to seamlessly blend with your digital ecosystem. This tool is all about augmenting your digital interactions, enhancing productivity, and enabling a more intelligent, GPT-powered experience in every aspect of your online presence.
-
-## Features
-
-1. Text Analysis: Easily extract summaries from texts.
-2. Clipboard Integration: Conveniently copy responses to the clipboard.
-3. File Output: Save responses to files for later reference.
-4. Pattern Module: Utilize specific modules for different types of analysis.
-5. Server Mode: Operate the tool in server mode for expanded capabilities.
-6. Remote & Standalone Modes: Choose between remote and standalone operations.
-
-## Installation
-
-1. Install dependencies:
- `npm install`
-2. Start the application:
- `npm start`
-
-Contributing
-
-We welcome contributions to Fabric! For details on our code of conduct and the process for submitting pull requests, please read the CONTRIBUTING.md.
diff --git a/installer/client/gui/index.html b/installer/client/gui/index.html
deleted file mode 100644
index 9c539e7..0000000
--- a/installer/client/gui/index.html
+++ /dev/null
@@ -1,156 +0,0 @@
-
-
-
-
-
- Fabric
-
-
-
-
-
-
-