Update docs

README_en.md

@@ -62,19 +62,51 @@
      The default Embedding model [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese/tree/main) in this project occupies around 3GB of video memory and can also be configured to run on a CPU.
 ### Software Requirements
 
-This project has been tested in a Python 3.8 environment.
-### 1. Install Python Dependencies
+This repository has been tested with Python 3.8 and CUDA 11.7 environments.
 
-```commandline
-pip install -r requirements.txt
+### 1. Setting up the environment
+
+* Environment check
+
+```shell
+# First, make sure your machine has Python 3.8 or higher installed
+$ python --version
+Python 3.8.13
+
+# If your version is lower, you can use conda to install the environment
+$ conda create -p /your_path/env_name python=3.8
+
+# Activate the environment
+$ source activate /your_path/env_name
+
+# Deactivate the environment
+$ source deactivate /your_path/env_name
+
+# Remove the environment
+$ conda env remove -p  /your_path/env_name
+```
+
+* Project dependencies
+
+```shell
+
+# Clone the repository
+$ git clone https://github.com/imClumsyPanda/langchain-ChatGLM.git
+
+# Install dependencies
+$ pip install -r requirements.txt
 ```
-Note: When using `langchain.document_loaders.UnstructuredFileLoader` for unstructured file loading, you may need to install other dependent packages according to the documentation. Please refer to the [langchain documentation](https://python.langchain.com/en/latest/modules /indexes/document_loaders/examples/unstructured_file.html)
+
+Note: When using langchain.document_loaders.UnstructuredFileLoader for unstructured file integration, you may need to install other dependency packages according to the documentation. Please refer to [langchain documentation](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/unstructured_file.html).
 
 ### 2. Run Scripts to Experience Web UI or Command Line Interaction
+
 Execute [webui.py](webui.py) script to experience **Web interaction** <img src="https://img.shields.io/badge/Version-0.1-brightgreen">
 ```commandline
 python webui.py
 ```
+Note: Before executing, check the remaining space in the `$HOME/.cache/huggingface/` folder, at least 15G.
+
 The resulting interface is shown below:
 ![webui](img/ui1.png)
 The API interface provided in the Web UI is shown below:
@@ -92,6 +124,7 @@ python knowledge_based_chatglm.py
 ```
 
 ### FAQ
+
 Q1: What file formats does this project support?
 
 A1: Currently, this project has been tested with txt, docx, and md file formats. For more file formats, please refer to the [langchain documentation](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/unstructured_file.html). It is known that if the document contains special characters, there might be issues with loading the file.
@@ -120,6 +153,8 @@ Q5: Can this project run in Google Colab?
 
 A5: You can try running the chatglm-6b-int4 model in Google Colab. Please note that if you want to run the Web UI in Colab, you need to set the `share` parameter in `demo.queue(concurrency_count=3).launch(server_name='0.0.0.0', share=False, inbrowser=False)` to `True`.
 
+This issue is related to the system environment. For more details, please refer to [Issues with installing packages using pip in Anaconda](docs/Issue-with-Installing-Packages-Using-pip-in-Anaconda.md).
+
 ## DEMO
 
 Using the question "What are the limitations of ChatGLM-6B and how can they be improved?" as an example:
@@ -153,6 +188,7 @@ ChatGLM's answer after using LangChain to access the README.md file of the ChatG
 >5. Enhance the model architecture: Improve ChatGLM-6B's model architecture to boost its performance and capabilities. For example, employ larger neural networks or refined convolutional neural network structures.
 
 ## Road map
+
 - [x] Implement LangChain + ChatGLM-6B for local knowledge application
 - [x] Unstructured file access based on langchain
    - [x].md

docs/Issue-with-Installing-Packages-Using-pip-in-Anaconda.md

@@ -0,0 +1,114 @@
+## Issue with Installing Packages Using pip in Anaconda
+
+## Problem
+
+Recently, when running open-source code, I encountered an issue: after creating a virtual environment with conda and switching to the new environment, using pip to install packages would be "ineffective." Here, "ineffective" means that the packages installed with pip are not in this new environment.
+
+------
+
+## Analysis
+
+1. First, create a test environment called test: `conda create -n test`
+2. Activate the test environment: `conda activate test`
+3. Use pip to install numpy: `pip install numpy`. You'll find that numpy already exists in the default environment.
+
+```powershell
+Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
+Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packages (1.20.3)
+```
+
+4. Check the information of pip: `pip show pip`
+
+```powershell
+Name: pip
+Version: 21.2.4
+Summary: The PyPA recommended tool for installing Python packages.
+Home-page: https://pip.pypa.io/
+Author: The pip developers
+Author-email: distutils-sig@python.org
+License: MIT
+Location: c:\programdata\anaconda3\lib\site-packages
+Requires:
+Required-by:
+```
+
+5. We can see that the current pip is in the default conda environment. This explains why the package is not in the new virtual environment when we directly use pip to install packages - because the pip being used belongs to the default environment, the installed package either already exists or is installed directly into the default environment.
+
+------
+
+## Solution
+
+1. We can directly use the conda command to install new packages, but sometimes conda may not have certain packages/libraries, so we still need to use pip to install.
+2. We can first use the conda command to install the pip package for the current virtual environment, and then use pip to install new packages.
+
+```powershell
+# Use conda to install the pip package
+(test) PS C:\Users\Administrator> conda install pip
+Collecting package metadata (current_repodata.json): done
+Solving environment: done
+....
+done
+
+# Display the information of the current pip, and find that pip is in the test environment
+(test) PS C:\Users\Administrator> pip show pip
+Name: pip
+Version: 21.2.4
+Summary: The PyPA recommended tool for installing Python packages.
+Home-page: https://pip.pypa.io/
+Author: The pip developers
+Author-email: distutils-sig@python.org
+License: MIT
+Location: c:\programdata\anaconda3\envs\test\lib\site-packages
+Requires:
+Required-by:
+
+# Now use pip to install the numpy package, and it is installed successfully
+(test) PS C:\Users\Administrator> pip install numpy
+Looking in indexes: 
+https://pypi.tuna.tsinghua.edu.cn/simple
+Collecting numpy
+  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/4b/23/140ec5a509d992fe39db17200e96c00fd29603c1531ce633ef93dbad5e9e/numpy-1.22.2-cp39-cp39-win_amd64.whl (14.7 MB)
+Installing collected packages: numpy
+Successfully installed numpy-1.22.2
+
+# Use pip list to view the currently installed packages, no problem
+(test) PS C:\Users\Administrator> pip list
+Package      Version
+------------ ---------
+certifi      2021.10.8
+numpy        1.22.2
+pip          21.2.4
+setuptools   58.0.4
+wheel        0.37.1
+wincertstore 0.2
+```
+
+## Supplement
+
+1. The reason I didn't notice this problem before might be because the packages installed in the virtual environment were of a specific version, which overwrote the packages in the default environment. The main issue was actually a lack of careful observation:), otherwise, I could have noticed `Successfully uninstalled numpy-xxx` **default version** and `Successfully installed numpy-1.20.3` **specified version**.
+2. During testing, I found that if the Python version is specified when creating a new package, there shouldn't be this issue. I guess this is because pip will be installed in the virtual environment, while in our case, including pip, no packages were installed, so the default environment's pip was used.
+3. There's a question: I should have specified the Python version when creating a new virtual environment before, but I still used the default environment's pip package. However, I just couldn't reproduce the issue successfully on two different machines, which led to the second point mentioned above.
+4. After encountering the problem mentioned in point 3, I solved it by using `python -m pip install package-name`, adding `python -m` before pip. As for why, you can refer to the answer on [StackOverflow](https://stackoverflow.com/questions/41060382/using-pip-to-install-packages-to-anaconda-environment):
+
+>1. If you have a non-conda pip as your default pip but conda python as your default python (as below):
+>
+>```shell
+>>which -a pip
+>/home/<user>/.local/bin/pip   
+>/home/<user>/.conda/envs/newenv/bin/pip
+>/usr/bin/pip
+>
+>>which -a python
+>/home/<user>/.conda/envs/newenv/bin/python
+>/usr/bin/python
+>```
+>
+>2. Then, instead of calling `pip install <package>` directly, you can use the module flag -m in python so that it installs with the anaconda python
+>
+>```shell
+>python -m pip install <package>
+>```
+>
+>3. This will install the package to the anaconda library directory rather than the library directory associated with the (non-anaconda) pip
+>4. The reason for doing this is as follows: the pip command references a specific pip file/shortcut (which -a pip will tell you which one). Similarly, the python command references a specific python file (which -a python will tell you which one). For one reason or another, these two commands can become out of sync, so your "default" pip is in a different folder than your default python and therefore is associated with different versions of python.
+>5. In contrast, the python -m pip construct does not use the shortcut that the pip command points to. Instead, it asks python to find its pip version and use that version to install a package.

docs/在Anaconda中使用pip安装包无效问题.md

@@ -14,18 +14,14 @@
 
 3、使用 pip 安装 numpy，`pip install numpy`，会发现 numpy 已经存在默认的环境中
 
-```shell
+```powershell
 Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
 Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packages (1.20.3)
 ```
 
-Bash
-
-Copy
-
 4、这时候看一下 pip 的信息，`pip show pip`
 
-```shell
+```powershell
 Name: pip
 Version: 21.2.4
 Summary: The PyPA recommended tool for installing Python packages.
@@ -38,10 +34,6 @@ Requires:
 Required-by:
 ```
 
-Bash
-
-Copy
-
 5、可以发现当前 pip 是在默认的 conda 环境中。这也就解释了当我们直接使用 pip 安装包时为什么包不在这个新的虚拟环境中，因为使用的 pip 属于默认环境，安装的包要么已经存在，要么直接装到默认环境中去了。
 
 ------
@@ -52,7 +44,7 @@ Copy
 
 2、我们可以先使用 conda 命令为当前虚拟环境安装 pip 包，再使用 pip 安装新的包
 
-```shell
+```powershell
 # 使用 conda 安装 pip 包
 (test) PS C:\Users\Administrator> conda install pip
 Collecting package metadata (current_repodata.json): done
@@ -93,10 +85,6 @@ wheel        0.37.1
 wincertstore 0.2
 ```
 
-Bash
-
-Copy
-
 ------
 
 ## 补充
@@ -122,22 +110,14 @@ Copy
 > /usr/bin/python
 > ```
 >
-> Bash
->
-> Copy
->
 > 2、然后，而不是直接调用 `pip install <package>`，你可以在 python 中使用模块标志 -m，以便它使用 anaconda python 进行安装
 >
 > ```shell
-> python -m pip install <package>
+>python -m pip install <package>
 > ```
 >
-> Bash
->
-> Copy
->
 > 3、这将把包安装到 anaconda 库目录，而不是与（非anaconda） pip 关联的库目录
->
+> 
 > 4、这样做的原因如下：命令 pip 引用了一个特定的 pip 文件 / 快捷方式（which -a pip 会告诉你是哪一个）。类似地，命令 python 引用一个特定的 python 文件（which -a python 会告诉你是哪个）。由于这样或那样的原因，这两个命令可能变得不同步，因此你的“默认” pip 与你的默认 python 位于不同的文件夹中，因此与不同版本的 python 相关联。
 >
 > 5、与此相反，python -m pip 构造不使用 pip 命令指向的快捷方式。相反，它要求 python 找到它的pip 版本，并使用该版本安装一个包。