探索性数据分析

Exploratory Data Analysis

在本地检查 CSV、TSV、JSON 等数据的字段、缺失值、分布、异常值和数据质量。

灵栈官方最新版本 0.1.0更新于 2026-10-16下载 2 次

CSV
数据质量
缺失值
异常值
数据审计
科研数据

references/chemistry_molecular_formats.md

1	# Chemistry and Molecular Formats
2
3	Reviewed: 2026-09-30
4	Executable scope: No chemistry-native format has a bundled parser. This file
5	is a reference-only routing guide, not a support claim.
6
7	## Capability boundary
8
9	\| Format family \| Bundled chemistry inspection \| Required approach \|
10	\|---\|---\|---\|
11	\| PDB, PDBx/mmCIF/CIF \| No \| Dictionary/version-aware structural tooling \|
12	\| Molfile/SDF, SMILES, XYZ \| No \| Chemistry-aware parser with explicit sanitization policy \|
13	\| DCD/XTC/TRR and topology files \| No \| Topology-aware trajectory tooling \|
14	\| Gaussian/QM outputs, cube grids \| No \| Program/version-aware parser \|
15	\| Pickle/joblib/dill molecule/model files \| Never \| Obtain a non-executable interchange export \|
16	\| Genuine CSV/TSV/JSON/NPY/NPZ/HDF5 exports \| General inspector only \| Apply the exact general-format capability; no chemical semantics are inferred \|
17
18	The `.cif`, `.log`, `.out`, `.raw`, and `.dat` suffixes are ambiguous. The
19	capability manifest registers `.cif` and `.raw` as reference-only; unregistered
20	`.log`, `.out`, and `.dat` fail closed. It does not sniff content or guess a producer.
21
22	## PDB and PDBx/mmCIF
23
24	wwPDB states that PDBx/mmCIF is its official working and archive format.
25	Legacy PDB format 3.30 remains distributed where representable but has field
26	and size limitations.
27
28	Use a pinned parser such as Gemmi, Biopython's `Bio.PDB`, or official wwPDB
29	validation services/tools in a separately reviewed environment. Confirm:
30
31	- file/dictionary version and experimental method;
32	- model count, chain/entity mapping, assemblies, alternate locations,
33	insertion codes, occupancy, B factors, and missing residues/atoms;
34	- unit cell, symmetry, resolution, R factors, validation metrics, and
35	biological versus crystallographic assembly;
36	- ligand/component definitions, covalent links, protonation/charge assumptions,
37	and coordinate units; and
38	- whether multiple models are alternatives, an ensemble, or time/order data.
39
40	Do not interpret a low B factor, occupancy, model score, or missing atom as a
41	quality verdict without experimental context. Do not claim binding, stability,
42	function, or causality from a coordinate inventory.
43
44	## Molfile, SDF, and line notations
45
46	Molfile/SDF records can represent atoms, bonds, coordinates, charges,
47	stereochemistry, query features, and arbitrary property blocks. SMILES is a
48	line notation whose interpretation depends on aromaticity, valence,
49	stereochemistry, isotope, charge, and sanitization rules.
50
51	Before EDA:
52
53	1. Identify CTfile/version and producer.
54	2. Parse with errors preserved; count invalid records rather than silently
55	dropping them.
56	3. Keep the original string/record and a separate standardized representation.
57	4. Record sanitization, aromaticity, tautomer, protonation, salt/fragment,
58	stereochemistry, isotope, and charge policies.
59	5. Distinguish 2-D drawing coordinates from experimentally or computationally
60	meaningful 3-D conformers.
61	6. Treat property names/values as untrusted metadata and redact identifiers.
62
63	Descriptor distributions are conditional on these choices. Do not automatically
64	neutralize, desalinate, canonicalize, deduplicate, generate conformers, or
65	discard parser failures.
66
67	## XYZ and coordinate text
68
69	XYZ commonly starts each frame with atom count and a comment line, followed by
70	element and Cartesian coordinates. Variants can contain trajectories,
71	additional columns, or nonstandard units. Confirm:
72
73	- atom-count/frame boundaries;
74	- element/isotope labels and units (often Å, but not guaranteed);
75	- periodic cell/charge/spin information stored elsewhere;
76	- whether frames are independent molecules, optimization steps, or dynamics;
77	and
78	- topology/bond inference policy.
79
80	The generic tabular scanner is not an XYZ parser.
81
82	## Molecular dynamics trajectories
83
84	DCD, XTC, TRR, NetCDF trajectories, and related files usually need a matching
85	topology and sometimes unit-cell/time metadata. A suffix does not supply these.
86	With MDAnalysis/MDTraj or another pinned reader, inspect:
87
88	- topology/trajectory atom count and ordering;
89	- frame count, time step, units, coordinates, velocities/forces, and box;
90	- periodic-boundary and imaging/unwrapping choices;
91	- equilibration, sampling interval, restraints, thermostat/barostat, and
92	replica identity; and
93	- corrupted/truncated frames before calculating RMSD/RMSF or contacts.
94
95	Frames are temporally dependent. Do not treat frames as independent replicates
96	or split adjacent frames randomly across train/test.
97
98	## Quantum chemistry outputs and grids
99
100	`.log`/`.out` files are program- and version-specific; use cclib or a
101	producer-specific parser only after confirming the producer. Check:
102
103	- method, basis set, charge, multiplicity, units, software/version, and job
104	termination;
105	- optimization/frequency convergence and imaginary modes;
106	- geometry/energy step count and whether the final structure is intended;
107	- SCF convergence, warnings, symmetry, solvation, and corrections; and
108	- whether values are raw, relative, thermal-corrected, or post-processed.
109
110	Cube and similar volumetric grids require origin, axis vectors, shape, units,
111	orbital/density identity, and integration conventions. Bound grid reads and do
112	not eagerly load an unverified declared shape.
113
114	## HDF5, NumPy, and tabular chemistry exports
115
116	If the file is genuinely `.npy`, `.npz`, `.h5`, `.hdf5`, `.csv`, `.tsv`, or
117	strict `.json`, the general inspector can report container structure and
118	aggregate numeric properties. It cannot infer:
119
120	- atom/molecule/conformer axes;
121	- coordinate or energy units;
122	- descriptor definitions;
123	- train/test compound grouping;
124	- assay censoring or detection limits; or
125	- chemical identity from field names.
126
127	HDF5 object names/attributes are redacted, external/soft links are not followed,
128	and dataset values are not read. NumPy object arrays are rejected. Pickled
129	models or RDKit objects are never deserialized.
130
131	## Chemistry EDA rigor
132
133	1. Define the independent unit: compound, batch, conformer, frame, calculation,
134	assay plate, specimen, or replicate.
135	2. Preserve raw structures and measured values; record standardization as a
136	derived transformation.
137	3. Create a data dictionary with units, assay endpoints, bounds, censoring,
138	LOD/LOQ, qualifiers, and provenance.
139	4. Distinguish missing, failed, inactive, below detection, above quantitation,
140	and structurally invalid records.
141	5. Split related analogues, scaffolds, batches, time, sites, or subjects before
142	learned preprocessing to prevent leakage. Random row splits can be
143	misleading.
144	6. Compare robust/classical summaries and investigate outliers against
145	measurement and structure; do not delete automatically.
146	7. Treat transformations (for example log concentration) as scientifically
147	defined and retain units/inverse interpretation.
148	8. Label descriptor/property screening as exploratory and define multiplicity
149	control for inferential follow-up.
150	9. Do not infer binding, efficacy, toxicity, mechanism, or causal effects from
151	EDA alone.
152
153	## Recommended reference-only tooling
154
155	Pin and validate tooling per project rather than treating this list as bundled
156	support:
157
158	- Gemmi or Biopython for PDBx/mmCIF/PDB;
159	- RDKit or Open Babel for Molfile/SDF/SMILES;
160	- ASE for XYZ and computational structures;
161	- MDAnalysis or MDTraj for topology/trajectory pairs; and
162	- cclib for supported quantum-chemistry outputs.
163
164	Check each parser's current format table and release notes. Never pass untrusted
165	property text to shell commands or dynamic evaluation.
166
167	## Authoritative sources
168
169	Current parser APIs and format guidance reviewed 2026-09-30. Historical
170	methodology citations are retained as source context.
171
172	- wwPDB, [File Formats and the PDB](https://www.wwpdb.org/documentation/file-formats-and-the-pdb)
173	(PDBx/mmCIF is the official archive/working format; legacy PDB format 3.30
174	where representable).
175	- wwPDB, [PDBx/mmCIF Dictionary Resources](https://mmcif.wwpdb.org/) and
176	[current user guide](https://mmcif.wwpdb.org/docs/user-guide/guide.html).
177	- wwPDB, [legacy PDB format 3.30](https://www.wwpdb.org/documentation/file-format-content/format33/v3.3.html).
178	- IUCr, [CIF format specifications](https://www.iucr.org/resources/cif/spec)
179	(links to CIF 1.1 and 2.0 syntax).
180	- RDKit, [current file parsing API](https://www.rdkit.org/docs/GettingStartedInPython.html#reading-and-writing-molecules).
181	- MDAnalysis, [supported topology and trajectory formats](https://userguide.mdanalysis.org/stable/formats/index.html).
182	- cclib, [supported programs and data](https://cclib.github.io/data.html).
183	- NIST/SEMATECH, [Exploratory Data Analysis](https://www.itl.nist.gov/div898/handbook/eda/eda.htm).
184	- scikit-learn, [data leakage guidance](https://scikit-learn.org/stable/common_pitfalls.html).

文件内容是平台检查时读出的文本，只供查看。平台不会运行包里的任何脚本或程序。