Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Documentation, Prepare for Desktop Build #11

Merged
merged 11 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v18.19.0
v18.19.1
9 changes: 7 additions & 2 deletions DEVELOPERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ Settings can be configured locally through `settings/local.js` — care should b
## Overview
The project is primarily built in JavaScript, running Node.js on the server and leveraging React on the client side. The client is transpiled using Webpack, and delivered as a complete bundle to the `assets/` directory. This directory can be served by a static web server, so long as update operations (and requests for JSON representations of hosted resources) are passed through to the backend HTTP server (served on port `3045` by default).

### Build Process
Running `npm start` will compile the UI using `scripts/build.js` to the `assets/` directory. You can serve this directory from any standard web server, but you will need to route HTTP requests with the `Accept: application/json` header to the backend server (port 3045) in addition to WebSockets if you want real-time functionality.

You can run the node without compiling the UI using `scripts/node.js` — this can aide in accelerating server-side development.

### Breakdown
- Coordinator — `scripts/node.js` the Node.js master process, managing:
- Sensemaker — `services/sensemaker.js` implements Fabric Service
Expand Down Expand Up @@ -53,8 +58,8 @@ LangChain is available through `services/trainer.js` which also handles all gene
- Ollama is used to provide a standard API for interfacing with LLMs
- Fabric is used for connectivity between instances

## Python Environment
Run `source .env/bin/activate` to enter the Python environment. See also `requirements.txt` for dependencies.
## Design & CSS
We use a custom Semantic UI theme, located in `libraries/semantic` — you can modify the theme and recompile the CSS styles using the `npm run build:semantic` command.

## Tips
- You can use `scripts/node.js` to quickly run the service without building: `node scripts/node.js`
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:18.19.0-bookworm
FROM node:18.19.1-bookworm

# Environment
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
Expand Down
4 changes: 2 additions & 2 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ cd sensemaker
./scripts/install-ollama.sh
./scripts/install-models.sh
./scripts/nvm/install.sh
nvm i 18.19.0
nvm i 18.19.1
```

### Redis Stack
Expand Down Expand Up @@ -79,7 +79,7 @@ sudo mkdir -p /media/storage/node/files
## MacOS
- Homebrew
- NVM: `./scripts/nvm/install.sh`
- Node: `nvm install 18.19.0` (double-check documentation)
- Node: `nvm install 18.19.1` (double-check documentation)
- `brew install python3 pkg-config pixman cairo pango`
- `ssh-keygen -t ed25519`
- PUPPETEER_SKIP_DOWNLOAD=true npm run report:install
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ making sense of stuff.

**`SENSEMAKER`** provides robust intelligence gathering services, accumulating and organizing information from a set of configurable data streams to automatically construct new models and foster insight into complex systems.

## Requirements
- Node 18.19 (use `scripts/nvm/install.sh`)
- MySQL 8.3
- Redis Stack 7.2

## Quick Start
From the repository root:
```
Expand All @@ -18,17 +23,17 @@ Your username and password will be displayed in the logs **for the first boot on
Enjoy!

## Configuring
Settings may be provided by modifying `settings/local.js`
Settings may be provided by modifying `settings/local.js` with any of the following properties:

```yaml
alias: Network alias. Used when connecting to peers.
fabric: Fabric configuration.
seed: Seed phrase.
```

## Architecture
### Fabric
Sensemaker searches [the Fabric Network][fabric-pub] to aggregate information from a variety of sources.

### Project Structure
The repository is configured as follows:
```
Expand Down
16 changes: 15 additions & 1 deletion actions/apiActions.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

const fetch = require('node-fetch');

async function fetchFromAPI (path, params = {},token = null) {
async function fetchFromAPI (path, params = {}, token = null) {
const response = await fetch(path, {
method: 'GET',
headers: {
Expand All @@ -15,6 +15,19 @@ async function fetchFromAPI (path, params = {},token = null) {
return await response.json();
}

async function fetchResource (path = document.path) {
const response = await fetch(path, {
method: 'GET',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': (token) ? `Bearer ${token}` : undefined
}
});

return response.json();
}

async function patchAPI (path, params, token = null) {
const response = await fetch(path, {
method: 'PATCH',
Expand Down Expand Up @@ -47,6 +60,7 @@ async function postAPI (path, params, token = null) {

module.exports = {
fetchFromAPI,
fetchResource,
patchAPI,
postAPI
};
40 changes: 37 additions & 3 deletions actions/chatActions.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ const GET_MESSAGES_REQUEST = 'GET_MESSAGES_REQUEST';
const GET_MESSAGES_SUCCESS = 'GET_MESSAGES_SUCCESS';
const GET_MESSAGES_FAILURE = 'GET_MESSAGES_FAILURE';

const FETCH_RESPONSE_REQUEST = 'FETCH_RESPONSE_REQUEST';
const FETCH_RESPONSE_SUCCESS = 'FETCH_RESPONSE_SUCCESS';
const FETCH_RESPONSE_FAILURE = 'FETCH_RESPONSE_FAILURE';

const GET_INFORMATION_REQUEST = 'GET_INFORMATION_REQUEST';
const GET_INFORMATION_SUCCESS = 'GET_INFORMATION_SUCCESS';
const GET_INFORMATION_FAILURE = 'GET_INFORMATION_FAILURE';
Expand Down Expand Up @@ -43,11 +47,9 @@ const resetChat = (message) => {
const submitMessage = (message, collection_id = null, file_fabric_id = null) => {
return async (dispatch, getState) => {
dispatch(messageRequest());

const token = getState().auth.token;

try {

let requestBody = { ...message };
if (file_fabric_id !== null) {
requestBody.file_fabric_id = file_fabric_id;
Expand Down Expand Up @@ -75,10 +77,38 @@ const submitMessage = (message, collection_id = null, file_fabric_id = null) =>
};
};

const regenAnswer = (message, collection_id = null, file_fabric_id = null) => {
const fetchResponse = (message) => {
return async (dispatch, getState) => {
dispatch(messageRequest());
const token = getState().auth.token;

try {
let requestBody = { ...message };
const response = await fetch('/v1/chat/completions', {
method: 'POST',
headers: {
'Authorization': `Bearer ${token}`,
'Content-Type': 'application/json',
},
body: JSON.stringify(requestBody)
});

if (!response.ok) {
const error = await response.json();
throw new Error(error.message);
}

const result = await response.json();
dispatch(messageSuccess(result));
} catch (error) {
dispatch(messageFailure(error.message));
}
};
};

const regenAnswer = (message, collection_id = null, file_fabric_id = null) => {
return async (dispatch, getState) => {
dispatch(messageRequest());
const token = getState().auth.token;

message.temperature = 'extreme';
Expand Down Expand Up @@ -178,6 +208,7 @@ const getMessageInformation = (request) => {
module.exports = {
resetChat,
submitMessage,
fetchResponse,
getMessages,
regenAnswer,
getMessageInformation,
Expand All @@ -187,6 +218,9 @@ module.exports = {
GET_MESSAGES_REQUEST,
GET_MESSAGES_SUCCESS,
GET_MESSAGES_FAILURE,
FETCH_RESPONSE_REQUEST,
FETCH_RESPONSE_SUCCESS,
FETCH_RESPONSE_FAILURE,
RESET_CHAT_STATE,
RESET_CHAT_SUCCESS
};
3 changes: 0 additions & 3 deletions actions/documentActions.js
Original file line number Diff line number Diff line change
Expand Up @@ -161,19 +161,16 @@ const uploadDocument = (file) => {
});

const fileCreation = await Promise.race([timeoutPromise, fetchPromise]);

if (!fileCreation.ok) {
const errorData = await fileCreation.json();
throw new Error(errorData.message || 'Server error');
}

const fileAnswer = await fileCreation.json();

dispatch(uploadDocumentSuccess(fileAnswer.fabric_id));
} catch (error) {
dispatch(uploadDocumentFailure(error.message));
}

}
}

Expand Down
1 change: 0 additions & 1 deletion actions/feedbackActions.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ const sendFeedbackRequest = () => ({ type: SEND_FEEDBACK_REQUEST });
const sendFeedbackSuccess = () => ({ type: SEND_FEEDBACK_SUCCESS });
const sendFeedbackFailure = (error) => ({ type: SEND_FEEDBACK_FAILURE, payload: error });


// Thunk action creator
const sendFeedback = (comment) => {
return async (dispatch, getState) => {
Expand Down
2 changes: 0 additions & 2 deletions actions/fileActions.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ const UPLOAD_FILE_REQUEST = 'UPLOAD_FILE_REQUEST';
const UPLOAD_FILE_SUCCESS = 'UPLOAD_FILE_SUCCESS';
const UPLOAD_FILE_FAILURE = 'UPLOAD_FILE_FAILURE';



// Action creators
const fetchFilesRequest = () => ({ type: FETCH_FILES_REQUEST, loading: true });
const fetchFilesSuccess = (files) => ({ type: FETCH_FILES_SUCCESS, payload: files, loading: false });
Expand Down
23 changes: 23 additions & 0 deletions actions/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ const {
const {
resetChat,
submitMessage,
fetchResponse,
regenAnswer,
getMessages,
getMessageInformation,
Expand Down Expand Up @@ -117,6 +118,13 @@ const {
searchGlobal,
} = require('../actions/searchActions');

// ## Task Actions
const {
createTask,
fetchTasks,
fetchTask
} = require('../actions/taskActions');

// ## Feedback Actions
const {
sendFeedback,
Expand All @@ -140,7 +148,18 @@ const {
clearQueue,
} = require('../actions/redisActions');

// ## Wallet Actions
const {
fetchKey,
fetchKeys,
createKey,
} = require('../actions/walletActions');

// ## Export
module.exports = {
fetchKey,
fetchKeys,
createKey,
fetchContract: fetchContract,
signContract: signContract,
fetchConversation: fetchConversation,
Expand All @@ -165,6 +184,7 @@ module.exports = {
fetchInquiries: fetchInquiries,
deleteInquiry: deleteInquiry,
createInquiry: createInquiry,
createTask: createTask,
fetchInvitation: fetchInvitation,
fetchInvitations: fetchInvitations,
sendInvitation: sendInvitation,
Expand All @@ -175,6 +195,9 @@ module.exports = {
deleteInvitation: deleteInvitation,
fetchPeople: fetchPeople,
fetchPerson: fetchPerson,
fetchResponse: fetchResponse,
fetchTask: fetchTask,
fetchTasks: fetchTasks,
fetchUploads,
fetchUpload,
searchUploads,
Expand Down
Loading
Loading