[FEATURE] Dashboard for AI Integration #143
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,4 +12,7 @@ docs | |
|
|
||
| # exclude aux files | ||
| .git | ||
| .idea | ||
|
|
||
| # exclude database dumps | ||
| db_dumps | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,7 @@ | |
| * | ||
| * @author Nils Dycke | ||
| */ | ||
|
|
||
| module.exports = { | ||
| development: { | ||
| username: 'postgres', | ||
|
|
||
|
Collaborator
There was a problem hiding this comment. why do all migration files have just a counter 00000 .... 00001 etc. usually it is the time when the file is created |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.sequelize.query('CREATE EXTENSION IF NOT EXISTS pgcrypto;'); | ||
|
|
||
| await queryInterface.createTable('ai_api_key', { | ||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| userId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'user', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'CASCADE', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| provider: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| name: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| apiEndpoint: { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| encryptedKey: { | ||
|
Collaborator
There was a problem hiding this comment. Do we need this here? We should encrypted the columns from the postgres table itself
Collaborator
There was a problem hiding this comment. Not sure if is resolved, I can't see a change in the code, please reference to the commit if comments are resolved or explain why it is resolved |
||
| type: Sequelize.BLOB, | ||
| allowNull: false, | ||
| }, | ||
| enabled: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: true, | ||
| }, | ||
| usageLimitMonthly: { | ||
|
Collaborator
There was a problem hiding this comment. Needs to be discussed how to limit the API keys and models (please set it on the agenda of the team meetings)
Collaborator
Author
There was a problem hiding this comment. Okay, I have set this point as one of the points to be discussed in our gradient meeting |
||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| lastUsedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('ai_api_key'); | ||
| }, | ||
| }; | ||
|
Collaborator
There was a problem hiding this comment. The sharing option should be represented in the model table. We should be able to share the model to a user or a group of users (we have the definition of roles in the database). You can ask @karimouf for some pointer on how to share it between different roles of users. This might need additional db tables, as we want to also define the limits for that sharing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.createTable('ai_model', { | ||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| name: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| modelPath: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| apiBaseUrl: { | ||
|
Collaborator
There was a problem hiding this comment. Please discuss with @akash9676 what parameters are needed for a model
Collaborator
Author
There was a problem hiding this comment. Okay @dennis-zyska. I have added this point in our internal meeting with @akash9676 |
||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| enabled: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: true, | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('ai_model'); | ||
| }, | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.createTable('ai_log', { | ||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| userId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'user', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'CASCADE', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| apiKeyId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| references: { | ||
| model: 'ai_api_key', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'SET NULL', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| modelId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'ai_model', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'CASCADE', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| documentId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| studySessionId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| studyStepId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| input: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| }, | ||
| output: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| }, | ||
| inputTokens: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| outputTokens: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| estimatedCost: { | ||
| type: Sequelize.FLOAT, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| latencyMs: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| status: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| defaultValue: 'success', | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
|
Collaborator
There was a problem hiding this comment. here the default columns are missing deletedAt, etc.
Collaborator
There was a problem hiding this comment. I guess this is resoveld, I can see the deletedAt now. |
||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('ai_log'); | ||
| }, | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.createTable('prompt_template', { | ||
|
Collaborator
There was a problem hiding this comment. The prompt templates are handled by our template integration, please ask @mohammadsherif0 how to use it in detail (we need to add another type of template for it, two types, one usual Prompt template and one that can be used in studies). so this table is probably not needed |
||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| userId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'user', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'CASCADE', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| name: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| description: { | ||
| type: Sequelize.TEXT, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| provider: { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| model: { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| promptText: { | ||
| type: Sequelize.TEXT, | ||
| allowNull: false, | ||
| }, | ||
| inputMapping: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| defaultValue: {}, | ||
| }, | ||
| defaultOutputMapping: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| defaultValue: {}, | ||
| }, | ||
| shared: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| sharedScope: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| defaultValue: 'none', | ||
| }, | ||
| sharedTargetId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('prompt_template'); | ||
| }, | ||
| }; | ||
|
Collaborator
There was a problem hiding this comment. Not needed, this is coming from the LiteLLM and brokerIO integration
Collaborator
There was a problem hiding this comment. Just delete the whole file |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| // Models are populated dynamically via LiteLLM and BrokerIO integration | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| }, | ||
| }; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is that needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was added so that when we run
npx sequelize-cli db:migrate, the CLI loads the .env file and has access to environment variables likePOSTGRES_HOST,POSTGRES_CAREDB, etc. Without it, theconfig.jscan't resolveprocess.env.POSTGRES_CAREDBand the migration fails.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dennis-zyska do we need that file though?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we usually don't use the command without the make file, so usually the envs are set. I wonder if it would always overwrite the current one with the .env (because we have multiple .env files), and the others would not work anymore