High volume data generation on top of MongoDB#105
Conversation
…erfile to expose port 8080, and add MongoDB service to Docker Compose
…add unit tests for settings functionality
…ations, update setup logic, and remove unused models
…pdate method signatures, and enhance test coverage
…Tests.ts run via "npx tsx src/__tests__/services/calendarClockServiceTests.ts"
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
ESLint found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Dependency ReviewThe following issues were found:
|
| }], | ||
| attributes: ['name', 'org', 'slug', 'description', 'html_url'] | ||
| const query = req.query.org ? { org: req.query.org as string } : {}; | ||
| const teams = await Team.find(query) |
Check failure
Code scanning / CodeQL
Database query built from user-controlled sources High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
To fix the problem, we need to ensure that the user input is properly sanitized before being used in the query. For MongoDB, we can use the $eq operator to ensure that the user input is interpreted as a literal value and not as a query object. This will prevent any potential NoSQL injection attacks.
We will modify the construction of the query object to use the $eq operator for the org field. This change will be made in the getAllTeams method of the TeamsController class.
| @@ -10,3 +10,3 @@ | ||
| try { | ||
| const query = req.query.org ? { org: req.query.org as string } : {}; | ||
| const query = req.query.org ? { org: { $eq: req.query.org as string } } : {}; | ||
| const teams = await Team.find(query) |
…pdate routes for adoption data retrieval
…rvice return types
…ice for unique ID generation, and enhance database connection settings
…dling, and clean up test files
There was a problem hiding this comment.
Copilot reviewed 60 out of 75 changed files in this pull request and generated 1 comment.
Files not reviewed (15)
- .vscode/tasks.json: Language not supported
- Dockerfile: Language not supported
- backend/github-manifest.json: Language not supported
- backend/package.json: Language not supported
- backend/src/tests/mock/metrics-gen/example.json: Language not supported
- backend/src/tests/mock/seats-gen/seatsExampleTest.json: Language not supported
- backend/src/tests/mock/survey-gen/exampleSurvey.json: Language not supported
- README.md: Evaluated as low risk
- backend/src/tests/mock/metrics-gen/runExampleMock.ts: Evaluated as low risk
- backend/jest.config.ts: Evaluated as low risk
- backend/src/tests/mock/metrics-gen/runMock.ts: Evaluated as low risk
- backend/src/tests/mock/metrics-gen/mockGenerator.ts: Evaluated as low risk
- backend/src/tests/mock/mock.mongo.ts: Evaluated as low risk
- backend/src/tests/mock/seats-gen/runSeatsGenerator.ts: Evaluated as low risk
- backend/src/tests/mock/seats-gen/mockSeatsGenerator.js: Evaluated as low risk
Comments suppressed due to low confidence (3)
backend/src/tests/mock/seats-gen/mockSeatsGenerator.ts:25
- The variable 'lastActivityAt' should have a specific type instead of 'any'. Consider using 'string' or 'Date'.
const lastActivityAt : any = seat.last_activity_at;
backend/src/tests/mock/seats-gen/mockSeatsGenerator.ts:55
- Use '===' instead of '==' for comparing dates.
if (newActivity == currentActivity ){
backend/src/tests/mock/seats-gen/mockSeatsGenerator.ts:74
- The property 'specificUser' is not defined in the 'SeatsMockConfig' type. Ensure it is part of the configuration.
seat.specificUser = this.config.specificUser;
Tip: If you use Visual Studio Code, you can request a review from Copilot before you push from the "Source Control" tab. Learn more
… creation logic; add Member interface in API service
… adjust sorting order in adoption service
… fields in exampleSurvey.json; refactor survey generator functions for clarity
…to use findOneAndUpdate; change data types from string to number in database schemas; adjust form inputs in value modeling component for consistency.
| }); | ||
| const updated = await Survey.findOneAndUpdate({ | ||
| id: Number(id) // Cast `id` to Number | ||
| }, req.body); |
Check failure
Code scanning / CodeQL
Database query built from user-controlled sources High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
To fix the problem, we need to ensure that the data from req.body is sanitized before it is used in the database query. One way to achieve this is by using the $set operator in the findOneAndUpdate method to ensure that only the fields we expect are updated. Additionally, we should validate the id parameter to ensure it is a valid identifier.
- Use the
$setoperator to update the fields in the document. - Validate the
idparameter to ensure it is a valid identifier. - Sanitize the
req.bodyobject to ensure it only contains the fields we expect to update.
| @@ -126,5 +126,6 @@ | ||
| const { id } = req.params; | ||
| const updated = await Survey.findOneAndUpdate({ | ||
| id: Number(id) // Cast `id` to Number | ||
| }, req.body); | ||
| const updated = await Survey.findOneAndUpdate( | ||
| { id: Number(id) }, // Cast `id` to Number | ||
| { $set: req.body } | ||
| ); | ||
| if (updated) { |
…e schema for assignee tracking
…thub-value into mongoose
…nd update table header to be disabled; enhance metrics service logging; modify seat service to include additional member details and update database schema for seat associations.
…ling logic and unnecessary console logs; clean up seat service activity updates and metrics service logging.
…unused form fields in the value component
…r and tests. disable components in value modeling and pick up the org change event.
…y to daily activity charts (if only one series is selected) enhance mock survey generator logic and adjust value modeling table layout.
| throw new Error('Invalid survey data provided'); | ||
| } | ||
| const Survey = mongoose.model('Survey'); | ||
| const result = await Survey.updateOne({ id: survey.id }, survey); |
Check failure
Code scanning / CodeQL
Database query built from user-controlled sources High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
To fix the problem, we need to ensure that user-provided data is properly sanitized before being used in a MongoDB query. We can use the $eq operator to ensure that the user input is interpreted as a literal value and not as a query object. Additionally, we should validate the survey object to ensure it does not contain any unexpected fields that could be used for injection.
- Modify the
updateSurveymethod inbackend/src/services/survey.service.tsto use the$eqoperator for theidfield. - Add validation to ensure that the
surveyobject only contains expected fields.
| @@ -17,4 +17,11 @@ | ||
| } | ||
| // Validate survey object to ensure it only contains expected fields | ||
| const validFields = ['id', 'status', 'reason', 'org', 'repo', 'prNumber', 'userId']; | ||
| for (const key of Object.keys(survey)) { | ||
| if (!validFields.includes(key)) { | ||
| throw new Error(`Unexpected field in survey data: ${key}`); | ||
| } | ||
| } | ||
| const Survey = mongoose.model('Survey'); | ||
| const result = await Survey.updateOne({ id: survey.id }, survey); | ||
| const result = await Survey.updateOne({ id: { $eq: survey.id } }, survey); | ||
|
|
2 participants
The calendarClockServiceTests.ts file simulates a calendar-clock function that runs various data generation tasks on an hourly basis. It connects to a MongoDB database and performs the following tasks:
Survey Generation: Randomly generates surveys 20% of the time during weekdays (Monday to Friday) between 6 AM and 11 PM.
Seats Generation: Generates seat data for each member of the team every hour.
Metrics Generation: Generates metrics data daily at 11 PM.
The script loops through each hour within a specified date range, incrementing the datetime parameter each cycle, and calls the respective data generation functions. It also retrieves all team members from the database to use in the seats generation process.