{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generating and managing inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{admonition} Learning Objectives\n",
    ":class: learning-objectives\n",
    "\n",
    "For larger and more complex projects it is beneficial to automate the generation of input nodes, rather than manually creating them.\n",
    "We would also like to be able to manage the inputs in a structured way, so that we can easily find them and change them.\n",
    "\n",
    "In this section we will look at methods for generating input nodes in a more automated fashion, and how to manage them using groups and extras.\n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous section, we generated the inputs for a calculation manually,\n",
    "by reading from a previously written input file.\n",
    "\n",
    "For more complex calculations though, it is likely that we would like to automate the input generation process.\n",
    "\n",
    "In this section, we will see the some methods for generating the inputs for a `PwCalculation` automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "hide-cell"
    ]
   },
   "outputs": [],
   "source": [
    "from local_module import load_temp_profile\n",
    "\n",
    "data = load_temp_profile(name=\"input-gen\", add_sssp=True, add_structure_si=True)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing structures from external databases\n",
    "\n",
    "AiiDA ships with a number of plugins that allow you to import structures from external databases.\n",
    "These plugins are called `importers` and are available in the `aiida.tools.dbimporters` module.\n",
    "\n",
    "As an example, let's import a structure from the [Crystallography Open Database (COD)](http://www.crystallography.net/cod/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aiida.tools.dbimporters.plugins.cod import CodEntry\n",
    "\n",
    "entry = CodEntry(\"http://www.crystallography.net/cod/1526655.cif\")\n",
    "structure = entry.get_aiida_structure()\n",
    "structure.get_pymatgen()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "AiiDA also works closely with the [Open Databases Integration for Materials Design (OPTIMADE) consortium](https://www.optimade.org/),\n",
    "which provides a common REST API for querying a number of databases for structures.\n",
    "\n",
    "![optimade](_static/aiida/optimade.png){align=center width=350px}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from local_module.optimade import count_structures, get_providers, yield_structures\n",
    "\n",
    "get_providers()[[\"name\", \"description\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use the OPTIMADE plugin to query the COD database for a structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count_structures(\"cod\", 'chemical_formula_hill = \"Si\" AND nelements = 1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for structure in yield_structures(\"cod\", 'chemical_formula_hill = \"Si\"', max_results=2):\n",
    "#     print(structure)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing Pseudopotential families"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [aiida-pseudo](https://github.com/aiidateam/aiida-pseudo) package provides a number of pseudopotential families that are available for download.\n",
    "These install a full set of pseudopotentials for all elements into the AiiDA profile as a `Group`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```console\n",
    "$ aiida-pseudo install sssp -x PBE -p efficiency -v 1.1\n",
    "```\n",
    "\n",
    "They can later be loaded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aiida import orm\n",
    "\n",
    "family = orm.load_group(label=\"SSSP/1.1/PBE/efficiency\")\n",
    "family"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Groups"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view all groups in our profile using the `verdi group list` command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%verdi group list -a --all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Groups are a powerful feature of AiiDA, allowing you to organize your data in a flexible way.\n",
    "They can be used to organize nodes into sets, or to assign nodes as members of subgroups.\n",
    "You can add any nodes to a group, and any node can be in multiple groups.\n",
    "\n",
    "Lets create a new group\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%verdi group create my_group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can add a set of structures to the group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "group = orm.load_group(\"my_group\")\n",
    "\n",
    "other_structure = data.structure.clone().store()\n",
    "\n",
    "for structure in [data.structure, other_structure]:\n",
    "    structure.store()\n",
    "    group.add_nodes(structure)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%verdi group show my_group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding extras to nodes and querying for them\n",
    "\n",
    "We can add extra information to nodes using the `extras` dictionary.\n",
    "This is a flexible way to add additional information to nodes that is not part of the node's provenance model,\n",
    "i.e. these can be modified after the node is stored (as opposed to other data)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "group = orm.load_group(\"my_group\")\n",
    "node: orm.Node\n",
    "for node in group.nodes:\n",
    "    node.base.extras.set('structure', 'silicon')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "AiiDA provides a powerful query system that allows you to search for nodes based on their properties,\n",
    "and location in the provenance graph.\n",
    "\n",
    "We can gather all the structures we tagged as `silicon` using the `QueryBuilder`:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = orm.QueryBuilder()\n",
    "query.append(orm.Group, filters={\"label\": \"my_group\"}, tag=\"group\")\n",
    "query.append(\n",
    "    orm.StructureData,\n",
    "    with_group=\"group\",\n",
    "    tag=\"structure\",\n",
    "    filters={\"extras.structure\": \"silicon\"},\n",
    "    project=\"*\",\n",
    ")\n",
    "query.all(flat=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{seealso}\n",
    "\n",
    "The [Query Builder documentation](https://aiida.readthedocs.io/projects/aiida-core/en/latest/howto/query.html) provides more information on how to use the `QueryBuilder`,\n",
    "and the features available.\n",
    "\n",
    ":::"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 ('aiida-qe-demo')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "20c30adb377910d9d5c8112cf74e9f7ecb37538254a701570d770f074373c53e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}