Faculty Finder

Ranking Methodology

Faculty are ranked by a two-stage LLM pipeline that evaluates alignment between each researcher's enriched profile and the funding opportunity requirements. The system does not assign investigator roles — those decisions belong to the research team.

Stage 1: Requirement Extraction

The uploaded document (or manually entered text) is sent to a large language model (via LiteLLM, temperature=0) which extracts structured requirements: grant title, funding agency, a 2–3 sentence summary, investigator requirements (expertise areas, qualifications, constraints per role), and overall research themes. The LLM is instructed to use the document's own terminology for roles and to never invent requirements not stated or clearly implied.

Stage 2: Faculty Matching & Scoring

A second LLM call (temperature=0.05) receives the extracted requirements alongside a compact summary of each faculty member's profile. For each candidate, it evaluates three dimensions on a 0–100 scale:

• Expertise alignment (highest weight) — How closely the faculty member's research interests, expertise keywords, and funded project history match the opportunity's required expertise areas and research themes.
• Methodological fit — Whether the researcher's methods (as inferred from publications, MeSH terms, and funded work descriptions) align with the methodological needs described in the opportunity.
• Track record — Strength of the researcher's publication count, h-index, and funding history relative to the opportunity's scope. A well-funded researcher with an active publication record in the relevant area scores higher.

The overall match score (0–100) is a weighted synthesis of these three dimensions, with expertise alignment weighted most heavily, followed by methodological fit, then track record. The LLM also generates a 2–3 sentence explanation for each match referencing specific overlaps between the researcher's profile and the opportunity requirements.

Filtering & Ranking Rules

• Only faculty with at least one of: research interests, enriched research interests, or expertise keywords are evaluated.
• Only matches with a score ≥ 40 are returned. At most 15 matches are shown.
• Matches are sorted by overall match score, descending.
• For large faculty sets (60+), a keyword pre-filter narrows candidates before the LLM stage. This pre-filter scores each faculty member's keyword overlap against the extracted requirement themes and expertise areas, then passes the top 60 candidates to the LLM. Faculty sets of 60 or fewer skip this step entirely.

What the LLM Sees Per Faculty Member

Each faculty member is represented as a single line: name, degrees, title, research interests (truncated to 300 characters from the enriched profile, falling back to original directory text), up to 8 expertise keywords, number of funded grants, and h-index. The LLM does not see full abstracts, publication lists, or grant amounts during matching — those details inform the enriched profile summaries that the LLM reads.

Faculty Profile Data Sources

Faculty profiles are constructed from the university directory and then enriched weekly (via automated pipeline) from six public academic data sources:

• UCSD Profiles / Scripps Profiles (confidence: 1.0) — Institutional source. Research descriptions and biography text scraped from profiles.ucsd.edu and scripps.ucsd.edu. Preferred over all other sources when there are conflicts.
• ORCID (confidence: 0.9) — Self-reported researcher data. Provides ORCID ID, publications, and funding records from the public ORCID API. Searched by name + UCSD affiliation, with a fallback name-only search.
• NIH RePORTER (confidence: 0.8) — Verified federal grant records. Queries the NIH RePORTER API for funded projects where the faculty member is a PI at UCSD. Returns grant titles, abstracts (500 char), amounts, dates, and co-PI names.
• NSF Awards (confidence: 0.8, SIO & Jacobs) — Verified federal grant records. Queries the NSF Award Search API for awards where the faculty member is PI at UCSD. Returns grant titles, program names, abstracts, and amounts.
• Semantic Scholar (confidence: 0.75) — Academic graph data. Provides h-index, total paper/citation counts, and the 20 most recent publications with journal and year. Author disambiguation uses UCSD affiliation matching.
• PubMed / NCBI E-utilities (confidence: 0.7) — Biomedical literature. Searches by author name + UCSD affiliation. Returns the 20 most recent publications with titles, journals, years, MeSH terms, and abstracts (500 char).

Enrichment & Normalization

Raw data from all sources is synthesized by an LLM normalizer into structured profile fields: a 2–4 sentence research narrative, expertise keywords, methodologies, disease areas (or research domains), and populations (or study systems). The normalizer merges and deduplicates across sources, preferring institutional sources when data conflicts. The original university directory text is preserved and never overwritten. Enrichment runs weekly via GitHub Actions, and every field change is recorded in an append-only audit log with source, confidence, old/new values, and timestamp.

Score Display

The colored bar represents the overall match score: green (≥ 80) indicates strong alignment, yellow (60–79) indicates moderate alignment, and red (< 60) indicates partial alignment. The three sub-scores (Expertise, Methods, Track Record) are shown individually below the bar. These are the raw LLM-assigned values — no post-processing, normalization, or re-weighting is applied after the model returns them.

Model Details

Both LLM calls use the same model configured via LiteLLM (default: openai/api-gpt-oss-120b). The extraction call uses temperature=0 for deterministic output; the matching call uses temperature=0.05 for near-deterministic results with minimal variation. JSON mode is requested when supported; the system falls back gracefully if the model doesn't support it. Each call includes retry logic (2 attempts with 1-second backoff) for parse failures.

Limitations

The model evaluates textual similarity and semantic relevance — it does not have access to private data, internal evaluations, or proprietary metrics. Name disambiguation in PubMed and Semantic Scholar can occasionally return publications by a different researcher with the same name. Results should be treated as a starting point for team assembly, not a definitive ranking or prediction of success.