DocuThinker-AI-App

DocuThinker - AI-Powered Document Analysis and Summarization App

Welcome to DocuThinker! This is a full-stack (FERN-Stack) application that integrates an AI-powered document processing backend with a React-based frontend. The app allows users to upload documents for summarization, generate key insights, and chat with an AI based on the document’s content.

DocuThinker Logo

📚 Table of Contents

📖 Overview

The DocuThinker app is designed to provide users with a simple, AI-powered document management tool. Users can upload PDFs or Word documents and receive summaries, key insights, and discussion points. Additionally, users can chat with an AI using the document’s content for further clarification.

DocuThinker is created using the FERN-Stack architecture, which stands for Firebase, Express, React, and Node.js. The backend is built with Node.js and Express, integrating Firebase for user authentication and MongoDB for data storage. The frontend is built with React and Material-UI, providing a responsive and user-friendly interface.

It is currently deployed live on Vercel and Render. You can access the live app here.

🚀 Live Deployments

We have deployed the entire app on Vercel and Render. You can access the live app here.

Note: The backend server may take a few seconds to wake up if it has been inactive for a while. The first API call may take a bit longer to respond. Subsequent calls should be faster as the server warms up.

Additionally, the app is currently on the Free Tier of Render, so it may take longer to process your request since we are only allocated 512MB and 0.1 CPU.

Live Statuses

Deployed with Vercel Render Success Netlify Backup Deployed Firebase Functional MongoDB Atlas Connected Redis Cache Enabled RabbitMQ Enabled GraphQL API Dockerized Kubernetes Jenkins CI/CD Swagger & OpenAPI

✨ Features

DocuThinker offers a wide range of features to help users manage and analyze their documents effectively. Here are some of the key features of the app:

⚙️ Technologies

React Material-UI TailwindCSS Craco Webpack Express Firebase Node.js GraphQL Google Cloud Natural Language API Google Speech-to-Text API RAG LangChain MongoDB Redis React Native Swagger OpenAPI Docker Kubernetes NGINX GitHub Actions Jenkins Render Vercel Netlify

🖼️ User Interface

DocuThinker features a clean and intuitive user interface designed to provide a seamless experience for users. The app supports both light and dark themes, responsive design, and easy navigation. Here are some screenshots of the app:

Landing Page

Landing Page

Landing Page - Dark Mode

Landing Page - Dark Mode

Document Upload Page

Document Upload Page

Document Upload Page - Dark Mode

Document Upload Page - Dark Mode

Document Upload Page - Document Uploaded

Document Upload Page - Document Uploaded

Google Drive Document Selection

Google Drive Document Selection

Home Page

Home Page

Home Page - Dark Mode

Home Page - Dark Mode

Chat Modal

Chat Modal

Chat Modal - Dark Mode

Chat Modal - Dark Mode

Documents Page

Documents Page

Documents Page - Dark Mode

Documents Page - Dark Mode

Document Page - Search Results

Document Page - Search Results

Profile Page

Profile Page

Profile Page - Dark Mode

Profile Page - Dark Mode

How To Use Page

How To Use Page

How To Use Page - Dark Mode

How To Use Page - Dark Mode

Login Page

Login Page

Login Page - Dark Mode

Login Page - Dark Mode

Registration Page

Registration Page

Registration Page - Dark Mode

Registration Page - Dark Mode

Forgot Password Page

Forgot Password Page

Forgot Password Page - Dark Mode

Forgot Password Page - Dark Mode

Responsive Design Example

Responsive Design

Navigation Drawer

404 Not Found Page

404 Not Found Page

404 Not Found Page - Dark Mode

404 Not Found Page - Dark Mode

Footer

📂 Complete File Structure

The DocuThinker app is organized into separate subdirectories for the frontend, backend, and mobile app. Each directory contains the necessary files and folders for the respective components of the app. Here is the complete file structure of the app:

DocuThinker-AI-App/
├── backend/
│   ├── ai_ml/
│   │   ├── perform_ner_pos.py        # Named Entity Recognition and Part-of-Speech Tagging
│   │   ├── sen_analysis.py           # Sentiment analysis for document text
│   │   ├── chat.js                   # Chatbot integration for AI chat functionality
│   │   ├── analyzer.js               # Document analyzer for generating key ideas and discussion points
│   │   ├── textStatistics.js         # Text statistics for analyzing document content
│   │   ├── documentClassifier.js     # Document classifier for categorizing documents
│   │   ├── summarizer.js             # Document summarizer for generating summaries
│   │   └── (and many more files...)  # Additional AI/ML services
│   ├── middleware/
│   │   └── jwt.js                    # Authentication middleware with JWT for the app's backend
│   ├── controllers/
│   │   └── controllers.js            # Controls the flow of data and logic
│   ├── graphql/
│   │   ├── resolvers.js              # Resolvers for querying data from the database
│   │   └── schema.js                 # GraphQL schema for querying data from the database
│   ├── models/
│   │   └── models.js                 # Data models for interacting with the database
│   ├── services/
│   │   └── services.js               # Models for interacting with database and AI/ML services
│   ├── views/
│   │   └── views.js                  # Output formatting for success and error responses
│   ├── redis/
│   │   └── redisClient.js            # Redis client for caching data in-memory
│   ├── swagger/
│   │   └── swagger.js                # Swagger documentation for API endpoints
│   ├── .env                          # Environment variables (git-ignored)
│   ├── firebase-admin-sdk.json       # Firebase Admin SDK credentials (git-ignored)
│   ├── index.js                      # Main entry point for the server
│   ├── Dockerfile                    # Docker configuration file
│   ├── manage_server.sh              # Shell script to manage and start the backend server
│   └── README.md                     # Backend README file
│
├── frontend/
│   ├── public/
│   │   ├── index.html                # Main HTML template
│   │   └── manifest.json             # Manifest for PWA settings
│   ├── src/
│   │   ├── assets/                   # Static assets like images and fonts
│   │   │   └── logo.png              # App logo or images
│   │   ├── components/
│   │   │   ├── ChatModal.js          # Chat modal component
│   │   │   ├── Spinner.js            # Loading spinner component
│   │   │   ├── UploadModal.js        # Document upload modal component
│   │   │   ├── Navbar.js             # Navigation bar component
│   │   │   ├── Footer.js             # Footer component
│   │   │   └── GoogleAnalytics.js    # Google Analytics integration component
│   │   ├── pages/
│   │   │   ├── Home.js               # Home page where documents are uploaded
│   │   │   ├── LandingPage.js        # Welcome and information page
│   │   │   ├── Login.js              # Login page
│   │   │   ├── Register.js           # Registration page
│   │   │   ├── ForgotPassword.js     # Forgot password page
│   │   │   └── HowToUse.js           # Page explaining how to use the app
│   │   ├── App.js                    # Main App component
│   │   ├── index.js                  # Entry point for the React app
│   │   ├── App.css                   # Global CSS 1
│   │   ├── index.css                 # Global CSS 2
│   │   ├── reportWebVitals.js        # Web Vitals reporting
│   │   ├── styles.css                # Custom styles for different components
│   │   └── config.js                 # Configuration file for environment variables
│   ├── .env                          # Environment variables file (e.g., REACT_APP_BACKEND_URL)
│   ├── package.json                  # Project dependencies and scripts
│   ├── craco.config.js               # Craco configuration file
│   ├── Dockerfile                    # Docker configuration file
│   ├── manage_frontend.sh            # Shell script for managing and starting the frontend
│   ├── README.md                     # Frontend README file
│   └── package.lock                  # Lock file for dependencies
│
├── mobile-app/                       # Mobile app directory
│   ├── app/                          # React Native app directory
│   ├── .env                          # Environment variables file for the mobile app
│   ├── app.json                      # Expo configuration file
│   ├── components/                   # Reusable components for the mobile app
│   ├── assets/                       # Static assets for the mobile app
│   ├── constants/                    # Constants for the mobile app
│   ├── hooks/                        # Custom hooks for the mobile app
│   ├── scripts/                      # Scripts for the mobile app
│   ├── babel.config.js               # Babel configuration file
│   ├── package.json                  # Project dependencies and scripts
│   └── tsconfig.json                 # TypeScript configuration file
│
├── kubernetes/                       # Kubernetes configuration files
│   ├── manifests/                    # Kubernetes manifests for deployment, service, and ingress
│   ├── backend-deployment.yaml       # Deployment configuration for the backend
│   ├── backend-service.yaml          # Service configuration for the backend
│   ├── frontend-deployment.yaml      # Deployment configuration for the frontend
│   ├── frontend-service.yaml         # Service configuration for the frontend
│   ├── firebase-deployment.yaml      # Deployment configuration for Firebase
│   ├── firebase-service.yaml         # Service configuration for Firebase
│   └── configmap.yaml                # ConfigMap configuration for environment variables
│
├── nginx/
│   ├── nginx.conf                    # NGINX configuration file for load balancing and caching
│   └── Dockerfile                    # Docker configuration file for NGINX
│
├── images/                           # Images for the README
├── .env                              # Environment variables file for the whole app
├── docker-compose.yml                # Docker Compose file for containerization
├── jsconfig.json                     # JavaScript configuration file
├── package.json                      # Project dependencies and scripts
├── package-lock.json                 # Lock file for dependencies
├── postcss.config.js                 # PostCSS configuration file
├── tailwind.config.js                # Tailwind CSS configuration file
├── render.yaml                       # Render configuration file
├── vercel.json                       # Vercel configuration file
├── openapi.yaml                      # OpenAPI specification for API documentation
├── manage_docuthinker.sh             # Shell script for managing and starting the app (both frontend & backend)
├── jenkins_cicd.sh                   # Shell script for managing the Jenkins CI/CD pipeline
├── .gitignore                        # Git ignore file
├── LICENSE.md                        # License file for the project
├── README.md                         # Comprehensive README for the whole app
└── (and many more files...)          # Additional files and directories not listed here

🛠️ Getting Started

Prerequisites

Ensure you have the following tools installed:

Additionally, basic fullstack development knowledge and AI/ML concepts are recommended to understand the app’s architecture and functionalities.

Frontend Installation

  1. Clone the repository:

    git clone https://github.com/hoangsonww/DocuThinker-AI-App.git
cd DocuThinker-AI-App/backend

  2. Navigate to the frontend directory:

    cd frontend

  3. Install dependencies:

    npm install

    Or npm install --legacy-peer-deps if you face any peer dependency issues.

  4. Start the Frontend React app: 
    npm start

  5. Build the Frontend React app (for production):

    npm run build
  6. Alternatively, you can use yarn to install dependencies and run the app: 
    yarn install
yarn start

  7. Or, for your convenience, if you have already installed the dependencies, you can directly run the app in the root directory using:

    npm run frontend

    This way, you don’t have to navigate to the frontend directory every time you want to run the app.

  8. The app’s frontend will run on http://localhost:3000. You can now access it in your browser.

Backend Installation

Note that this is optional since we are deploying the backend on Render. However, you can (and should) run the backend locally for development purposes.

  1. Navigate to the root (not backend) directory: 
    cd backend
  2. Install dependencies: 
    npm install

    Or npm install --legacy-peer-deps if you face any peer dependency issues.

  3. Start the backend server: 
    npm run server
  4. The backend server will run on http://localhost:3000. You can access the API endpoints in your browser or Postman.
  5. Additionally, the backend code is in the backend directory. Feel free to explore the API endpoints and controllers.

Note: Be sure to use Node v.20 or earlier to avoid compatibility issues with Firebase Admin SDK.

Running the Mobile App

  1. Navigate to the mobile app directory: 
    cd mobile-app
  2. Install dependencies: 
     npm install
  3. Start the Expo server: 
    npx expo start
  4. Run the app on an emulator or physical device: Follow the instructions in the terminal to run the app on an emulator or physical device.

📋 API Endpoints

The backend of DocuThinker provides several API endpoints for user authentication, document management, and AI-powered insights. These endpoints are used by the frontend to interact with the backend server:

Method Endpoint Description
POST /register Register a new user in Firebase Authentication and Firestore, saving their email and creation date.
POST /login Log in a user and return a custom token along with the user ID.
POST /upload Upload a document for summarization. If the user is logged in, the document is saved in Firestore.
POST /generate-key-ideas Generate key ideas from the document text.
POST /generate-discussion-points Generate discussion points from the document text.
POST /chat Chat with AI using the original document text as context.
POST /forgot-password Reset a user’s password in Firebase Authentication.
POST /verify-email Verify if a user’s email exists in Firestore.
GET /documents/{userId} Retrieve all documents associated with the given userId.
GET /documents/{userId}/{docId} Retrieve a specific document by userId and docId.
GET /document-details/{userId}/{docId} Retrieve document details (title, original text, summary) by userId and docId.
DELETE /delete-document/{userId}/{docId} Delete a specific document by userId and docId.
DELETE /delete-all-documents/{userId} Delete all documents associated with the given userId.
POST /update-email Update a user’s email in both Firebase Authentication and Firestore.
POST /update-password Update a user’s password in Firebase Authentication.
GET /days-since-joined/{userId} Get the number of days since the user associated with userId joined the service.
GET /document-count/{userId} Retrieve the number of documents associated with the given userId.
GET /user-email/{userId} Retrieve the email of a user associated with userId.
POST /update-document-title Update the title of a document in Firestore.
PUT /update-theme Update the theme of the app.
GET /user-joined-date/{userId} Get date when the user associated with userId joined the service.
GET /social-media/{userId} Get the social media links of the user associated with userId.
POST /update-social-media Update the social media links of the user associated with userId.
POST /update-profile Update the user’s profile information.
POST /update-document/{userId}/{docId} Update the document details in Firestore.
POST /update-document-summary Update the summary of a document in Firestore.
POST /sentiment-analysis Analyzes the sentiment of the provided document text
POST /bullet-summary Generates a summary of the document text in bullet points
POST /summary-in-language Generates a summary in the specified language
POST /content-rewriting Rewrites or rephrases the provided document text based on a style
POST /actionable-recommendations Generates actionable recommendations based on the document text
GET /graphql GraphQL endpoint for querying data from the database

More API endpoints will be added in the future to enhance the functionality of the app. Feel free to explore the existing endpoints and test them using Postman or Insomnia.

API Documentation

For example, our API endpoints documentation looks like this:

Swagger Documentation

Additionally, we also offer API file generation using OpenAPI. You can generate API files using the OpenAPI specification. Here is how:

npx openapi-generator-cli generate -i http://localhost:5000/api-docs -g typescript-fetch -o ./api

This will generate TypeScript files for the API endpoints in the api directory. Feel free to replace or modify the command as needed.

Using the openapi.yaml File

  1. View the API Documentation
  1. Test the API
  1. Generate Client Libraries
  1. Generate Server Stubs
  1. Run a Mock Server
  1. Validate the OpenAPI File

API Architecture

API Testing

Example Request to Register a User:

curl --location --request POST 'http://localhost:3000/register' \
--header 'Content-Type: application/json' \
--data-raw '{
    "email": "test@example.com",
    "password": "password123"
}'

Example Request to Upload a Document:

curl --location --request POST 'http://localhost:3000/upload' \
--header 'Authorization: Bearer <your-token>' \
--form 'File=@"/path/to/your/file.pdf"'

Error Handling

The backend APIs uses centralized error handling to capture and log errors. Responses for failed requests are returned with a proper status code and an error message:

{
  "error": "An internal error occurred",
  "details": "Error details go here"
}

🧰 GraphQL Integration

Introduction to GraphQL in Our Application

Our application supports a fully-featured GraphQL API that allows clients to interact with the backend using flexible queries and mutations. This API provides powerful features for retrieving and managing data such as users, documents, and related information.

Key Features of the GraphQL API

Getting Started

  1. GraphQL Endpoint:
    The GraphQL endpoint is available at: 
    https://docuthinker-ai-app.onrender.com/graphql

    Or, if you are running the backend locally, the endpoint will be:

    http://localhost:3000/graphql

  2. Testing the API:
    You can use the built-in GraphiQL Interface to test queries and mutations. Simply visit the endpoint in your browser. You should see the following interface:

    GraphiQL Interface

    Now you can start querying the API using the available fields and mutations. Examples are below for your reference.

Example Queries and Mutations

1. Fetch a User and Their Documents

This query retrieves a user’s email and their documents, including titles and summaries:

query GetUser {
  getUser(id: "USER_ID") {
    id
    email
    documents {
      id
      title
      summary
    }
  }
}

2. Fetch a Specific Document

Retrieve details of a document by its ID:

query GetDocument {
  getDocument(userId: "USER_ID", docId: "DOCUMENT_ID") {
    id
    title
    summary
    originalText
  }
}

3. Create a New User

Create a user with an email and password:

mutation CreateUser {
  createUser(email: "example@domain.com", password: "password123") {
    id
    email
  }
}

4. Update a Document Title

Change the title of a specific document:

mutation UpdateDocumentTitle {
  updateDocumentTitle(userId: "USER_ID", docId: "DOCUMENT_ID", title: ["Updated Title.pdf"]) {
    id
    title
  }
}

5. Delete a Document

Delete a document from a user’s account:

mutation DeleteDocument {
  deleteDocument(userId: "USER_ID", docId: "DOCUMENT_ID")
}

Advanced Tips

For more information about GraphQL, visit the official documentation. If you encounter any issues or have questions, feel free to open an issue in our repository.

📱 Mobile App

The DocuThinker mobile app is built using React Native and Expo. It provides a mobile-friendly interface for users to upload documents, generate summaries, and chat with an AI. The mobile app integrates with the backend API to provide a seamless experience across devices.

Currently, it is in development and will be released soon on both the App Store and Google Play Store.

Stay tuned for the release of the DocuThinker mobile app!

Below is a screenshot of the mobile app (in development):

Mobile App

📦 Containerization

The DocuThinker app can be containerized using Docker for easy deployment and scaling. Follow these steps to containerize the app:

  1. Run the following command to build the Docker image: 
    docker compose up --build
  2. The app will be containerized and ready to run on port 3000.

You can also view the image in the Docker Hub repository here.

🚧 Deployment

Frontend Deployment (Vercel)

  1. Install the Vercel CLI:

    npm install -g vercel

  2. Deploy the frontend:

    vercel

  3. Follow the instructions in your terminal to complete the deployment.

Backend Deployment (Render)

Important Note about Backend Deployment

⚖️ Load Balancing & Caching

🔗 Jenkins Integration

  1. Install Jenkins: 
    brew install jenkins
  2. Start Jenkins: 
    brew services start jenkins

  3. Access Jenkins: Open your browser and go to http://localhost:8080 to access the Jenkins dashboard.

  4. Follow the instructions to set up Jenkins and create a new pipeline.

If successful, you should see the Jenkins pipeline running and deploying the app automatically whenever changes are pushed to the main branch. Here is an example:

Jenkins Pipeline

🚢 Kubernetes Integration

🔧 Contributing

We welcome contributions from the community! Follow these steps to contribute:

  1. Fork the repository.

  2. Create a new branch: 
    git checkout -b feature/your-feature
  3. Commit your changes: 
    git commit -m "Add your feature"
  4. Push the changes: 
    git push origin feature/your-feature
  5. Submit a pull request: Please submit a pull request from your forked repository to the main repository. I will review your changes and merge them into the main branch shortly.

Thank you for contributing to DocuThinker! 🎉

📝 License

This project is licensed under the Creative Commons Attribution-NonCommercial License. See the LICENSE file for details.

The DocuThinker open-source project is for educational purposes only and should not be used for commercial applications. Feel free to use it for learning and personal projects!

📚 Additional Documentation

For more information on the DocuThinker app, please refer to the following resources:

👨‍💻 Author

Here are some information about me:

Happy Coding and Analyzing! 🚀

Created with ❤️ by Son Nguyen in 2024.

🔝 Back to Top

This site is open source. Improve this page.