VladPolskiy/nextcloud-documentation
1
0
Fork 0
mirror of https://github.com/nextcloud/documentation.git synced 2025-08-16 16:07:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #12182 from nextcloud/enh/admin/ai/overview

Browse Source 
Admin docs: Update AI docs
This commit is contained in:
Marcel Klehr
2024-09-10 09:57:47 +02:00
committed by GitHub
parent c3b80c9acc cc1b25a86e
commit 21ba1c580b
4 changed files with 42 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
admin_manual/ai/app_assistant.rst
View File

@ -43,10 +43,8 @@ Machine translation
In order to make use of machine translation features in the assistant, you will need an app that provides a translation backend:
* :ref:`translate<ai-app-translate>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request)
* :ref:`translate2 (ExApp)<ai-app-translate2>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request)
* *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported)
* *integration_libretranslate* - Integrates with the open source LibreTranslate API to provide translation functionality hosted commercially or on your own hardware (Only community supported)
Speech-To-Text
~~~~~~~~~~~~~~
@ -69,7 +67,7 @@ Text-To-Image
In order to make use of Text-To-Image features, you will need an app that provides an image generation backend:
* text2image_stablediffusion2 (Customer support available upon request)
* text2image_stablediffusion (Customer support available upon request)
* *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
* *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see :ref:`AI as a Service<ai-ai_as_a_service>`)

105
admin_manual/ai/app_translate.rst
View File

@ -1,105 +0,0 @@
==========================================
App: Local Machine translation (translate)
==========================================
.. _ai-app-translate:
The *translate* app is one of the apps that provide machine translation functionality in Nextcloud and act as a translation backend for the :ref:`Nextcloud Assistant app<ai-app-assistant>`, the *text* app and :ref:`other apps making use of the core Translation API<mt-consumer-apps>`. The *translate* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities.
The app currently supports the following languages:
* English (en)
* German (de)
* French (fr)
* Spanish (es)
* Chinese (zh)
As the models are entirely open source, the quality of translations may not be comparable to commercially available machine translation services.
Requirements
------------
* Minimal Nextcloud version: 26
* x86 CPU with 4-8 cores for the app to use (The more cores the faster it will be)
* 2GB of RAM for the app should be enough
* GNU lib C (musl is not supported)
* This app does not support using GPU for processing and may thus not be performing ideally for long texts
* The workload will run on the web server workers
(*Note*: Nextcloud AIO is currently not supported due to it using musl)
Space usage
~~~~~~~~~~~
* ~1GB per language pair
* ~10GB in total
Installation
------------
0. Make sure the :ref:`Nextcloud Assistant app<ai-app-assistant>` is installed
1. Install the *translate* app via the "Apps" page in Nextcloud, or by executing
occ app:enable translate
Setup
~~~~~
After installing this app you will need to run:
.. code-block::
occ translate:download-models
You may also download only specific languages by using the following command:
.. code-block::
occ translate:download-models <languages>
For example
.. code-block::
occ translate:download-models de en
will download both en->de and de->en.
.. code-block::
occ translate:download-models de en es
will download en->de, de->en, en->es, es->en, es->de, de->es
App store
---------
You can also find the app in our app store, where you can write a review: `<https://apps.nextcloud.com/apps/translate>`_
Repository
----------
You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: `<https://github.com/nextcloud/translate>`_
Nextcloud customers should file bugs directly with our Customer Support.
Ethical AI Rating
-----------------
Rating: 🟢
~~~~~~~~~~
Positive:
* the software for training and inference of this model is open source
* the trained model is freely available, and thus can be run on-premises
* the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.
Learn more about the Nextcloud Ethical AI Rating `in our blog <https://nextcloud.com/blog/nextcloud-ethical-ai-rating/>_`.
Known Limitations
-----------------
* Language models are likely to generate false information and should thus only be used in situations that are not critical. It's recommended to only use AI at the beginning of a creation process and not at the end, so that outputs of AI serve as a draft for example and not as final product. Always check the output of language models before using it.
* Make sure to test the translation model you are using it for whether it meets the use-case's quality requirements
* Language models notoriously have a high energy consumption
* Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying models. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI)

1
admin_manual/ai/index.rst
View File

@ -7,7 +7,6 @@ Artificial Intelligence
overview
app_assistant
app_translate
app_translate2
app_llm2
app_stt_whisper2

75
admin_manual/ai/overview.rst
View File

@ -22,13 +22,14 @@ Nextcloud uses modularity to separate raw AI functionality from the Graphical Us
"Suspicious login detection","`Suspicious Login <https://apps.nextcloud.com/apps/suspicious_login>`_","Green","Yes","Yes","Yes","Yes"
"Related resources","`Related Resources <https://apps.nextcloud.com/apps/related_resources>`_","Green","Yes","Yes","Yes","Yes"
"Recommended files","recommended_files","Green","Yes","Yes","Yes","Yes"
"Machine translation","`Translate <https://apps.nextcloud.com/apps/translate>`_","Green","Yes","Yes - Opus models by University Helsinki","Yes","Yes"
"","`Local Machine Translation 2 (ExApp) <https://apps.nextcloud.com/apps/translate2>`_","Green","Yes","Yes - MADLAD models by Google","Yes","Yes"
"","`LibreTranslate integration <https://apps.nextcloud.com/apps/integration_libretranslate>`_","Green","Yes","Yes - OpenNMT models","Yes","Yes"
"Text processing using LLMs","`llm2 (ExApp) <https://apps.nextcloud.com/apps/llm2>`_","Green","Yes","Yes - Llama 3.1 model by Meta","Yes","Yes"
"","`OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_","Red","No","No","No","No"
"","`OpenAI and LocalAI integration (via LocalAI) <https://apps.nextcloud.com/apps/integration_openai>`_","Yellow","Yes","Yes - e.g. Llama models by Meta", "No","Yes"
"Machine translation","`Local Machine Translation 2 (ExApp) <https://apps.nextcloud.com/apps/translate2>`_","Green","Yes","Yes - MADLAD models by Google","Yes","Yes"
"","`DeepL integration <https://apps.nextcloud.com/apps/integration_deepl>`_","Red","No","No","No","No"
"","`OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_","Red","No","No","No","No"
"","`OpenAI and LocalAI integration (via LocalAI) <https://apps.nextcloud.com/apps/integration_openai>`_","Green","Yes","Yes","Yes","Yes"
"","`Local Whisper Speech-To-Text 2 (ExApp) <https://apps.nextcloud.com/apps/stt_whisper2>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","Yes"
"Speech to Text","`Local Whisper Speech-To-Text 2 (ExApp) <https://apps.nextcloud.com/apps/stt_whisper2>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","Yes"
"","`OpenAI and LocalAI integration <https://apps.nextcloud.com/apps/integration_openai>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","No"
"","`Replicate integration <https://apps.nextcloud.com/apps/integration_replicate>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","No"
"Image generation","`Local Stable Diffusion <https://apps.nextcloud.com/apps/text2image_stablediffusion>`_","Yellow","Yes","Yes - StableDiffusion XL model by StabilityAI","No","Yes"
@ -67,6 +68,30 @@ Features used by other apps
Some of our AI features are realized as generic APIs that any app can use and any app can provide an implementation for by registering a provider. So far, these are
Machine translation, Speech-To-Text, Image generation, Text processing and Context Chat.
Text processing
^^^^^^^^^^^^^^^
.. _tp-consumer-apps:
As you can see in the table above we have multiple apps offering text processing using Large language models.
In downstream apps like Context Chat and assistant, users can use the text processing functionality regardless of which app implements it behind the scenes.
Frontend apps
~~~~~~~~~~~~~
* *Text* for offering an inline graphical UI for the various tasks
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ for offering a graphical UI for the various tasks and a smart picker
* `Mail <https://apps.nextcloud.com/apps/mail>`_ for summarizing mail threads (see :ref:`the Nextcloud Mail docs<mail_thread_summary>` for how to enable this)
* `Summary Bot <https://apps.nextcloud.com/apps/summarai>`_ for summarizing chat histories in `Talk <https://apps.nextcloud.com/apps/spreed>`_
Backend apps
~~~~~~~~~~~~
* :ref:`llm2<ai-app-llm2>` - Runs open source AI LLM models on your own server hardware (Customer support available upon request)
* `OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_ - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
Machine translation
^^^^^^^^^^^^^^^^^^^
@ -85,10 +110,8 @@ Frontend apps
Backend apps
~~~~~~~~~~~~
* :ref:`translate<ai-app-translate>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request)
* :ref:`translate2 (ExApp)<ai-app-translate2>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request)
* *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported)
* *integration_libretranslate* - Integrates with the open source LibreTranslate API to provide translation functionality hosted commercially or on your own hardware (Only community supported)
Speech-To-Text
^^^^^^^^^^^^^^
@ -101,37 +124,15 @@ Frontend apps
~~~~~~~~~~~~~
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ offering a graphical translation UI and a smart picker
* `Speech-to-Text Helper <https://apps.nextcloud.com/apps/stt_helper>`_ for providing a Speech-To-Text smart picker (deprecated; was merged into assistant)
* `Talk <https://apps.nextcloud.com/apps/spreed>`_ for transcribing calls (see `Nextcloud Talk docs <https://nextcloud-talk.readthedocs.io/en/latest/settings/#app-configuration>`_ for how to enable this)
Backend apps
~~~~~~~~~~~~
* :ref:`stt_whisper2<ai-app-stt_whisper2>` - Runs open source AI Speech-To-Text models on your own server hardware (Customer support available upon request)
* *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
* `OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_ - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
Text processing
^^^^^^^^^^^^^^^
.. _tp-consumer-apps:
As you can see in the table above we have multiple apps offering Text processing capabilities. In downstream apps like the Nextcloud Assistant app, users can use the text processing functionality regardless of which app implements it behind the scenes.
Frontend apps
~~~~~~~~~~~~~
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ for offering a graphical UI for the various tasks and a smart picker
* `GPTFreePrompt <https://apps.nextcloud.com/apps/gptfreeprompt>`_ for providing an llm smart picker (deprecated; was merged into assistant)
* `Mail <https://apps.nextcloud.com/apps/mail>`_ for summarizing mail threads (see :ref:`the Nextcloud Mail docs<mail_thread_summary>` for how to enable this)
* `Summary Bot <https://apps.nextcloud.com/apps/summarai>`_ for summarizing chat histories in `Talk <https://apps.nextcloud.com/apps/spreed>`_
Backend apps
~~~~~~~~~~~~
* :ref:`llm2<ai-app-llm2>` - Runs open source AI language models locally on your own server hardware (Customer support available upon request)
* *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
Image generation
^^^^^^^^^^^^^^^^
@ -143,18 +144,17 @@ Frontend apps
~~~~~~~~~~~~~
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ for offering a graphical UI and a smart picker
* `Text-to-Image Helper <https://apps.nextcloud.com/apps/stt_helper>`_ for providing a Text-to-Image smart picker (deprecated; was merged into assistant)
Backend apps
~~~~~~~~~~~~
* text2image_stablediffusion2 (Customer support available upon request)
* *integration_openai* - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
* text2image_stablediffusion (Customer support available upon request)
* `OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_ - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
* *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see :ref:`AI as a Service<ai-ai_as_a_service>`)
Context Chat (Tech preview)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Context Chat
^^^^^^^^^^^^
Our Context Chat feature was introduced in Nextcloud Hub 7 (v28). It allows asking questions to the assistant related to your documents in Nextcloud. You will need to install both the context_chat app as well as the context_chat_backend External App. Be prepared that things might break or be a little rough around the edges. We look forward to your feedback!
Frontend apps
@ -167,6 +167,13 @@ Backend apps
* :ref:`context_chat + context_chat_backend<ai-app-context_chat>` - (Customer support available upon request)
Provider apps
~~~~~~~~~~~~~
Apps can integrate their content with Context Chat to make it available for querying using Context Chat. The following apps have implemented this integration so far:
* *files*
* `Analytics <https://apps.nextcloud.com/apps/analytics>`_
Frequently Asked Questions
--------------------------