The OWASP Top 10 [1] is the probably the most well-known and recognised reference standard for the most critical web application security risks. This organisation has now started to working on creating a similar list for Large Language Models (LLM) Applications.
I'm posting about it here since I think it would be beneficial for safety alignment researchers to be involved for two reasons:
I have no idea about how many people within the AI safety and alignment community actually know about this initiative, but I did not find any reference to it on the EA forum or here on the alignment forum, so I thought I might as well post about it.
More information available here:
- https://owasp.org/www-project-top-10-for-large-language-model-applications/
- https://github.com/OWASP/www-project-top-10-for-large-language-model-applications
My favourite tip I rarely see mentioned in Anki discussions: add a hidden source field to your custom card template and paste the original source or reference or hyperlink it.
This is useful for several reasons:
- You can easily find the source to read more about the subject simply by editing the card in the app.
- You don't have to clutter the card with info about the source if it is not essential to the card.
- In addition to the source info, you can add your own notes about why this information was useful or why you chose this specific source etc. You can give additional context such as "This principle was explained to me by
... (read more)