Internal Medicine AI Evaluation Specialist

This role is for one of our clients
Compensation: $130-$180 per hour

Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.
In this role, you will develop realistic clinical scenarios, evaluate AI-generated responses against evidence-based standards, and help improve how AI systems reason about diagnosis, treatment planning, and patient management within Internal Medicine.
We welcome Internal Medicine attending physicians, hospitalists, fellows, final-year residents, and other qualified Internal Medicine professionals who are passionate about applying their expertise to the future of healthcare AI.
Requirements
Key Responsibilities
Design Clinical Scenarios

• Create realistic Internal Medicine case studies based on everyday clinical practice.

• Develop prompts covering:

• Diagnostic reasoning

• Differential diagnosis

• Treatment planning

• Risk stratification

• Evidence-based guideline adherence

- Long-term patient management

• Long-term patient management

• Produce attending physician-level reference solutions that reflect current clinical best practices.

- Ensure recommendations align with established evidence-based guidelines.

• Ensure recommendations align with established evidence-based guidelines.

• Review and assess AI-generated clinical responses using structured evaluation frameworks.

• Identify reasoning gaps, inaccuracies, omissions, and opportunities for improvement.

- Provide detailed written feedback to improve AI model performance and clinical reliability.

• Provide detailed written feedback to improve AI model performance and clinical reliability.

• Participate in onboarding sessions and specialty calibration meetings.

• Work with multidisciplinary teams to ensure consistency, accuracy, and quality across medical evaluations.

Required Qualifications
Applicants should meet one of the following criteria:
Attending Physicians

• Board-certified in Internal Medicine.

- Current, active, and unrestricted medical license.

• Current, active, and unrestricted medical license.

• Final-year Internal Medicine resident.

- Recent residency graduates must be board-eligible.

• Recent residency graduates must be board-eligible.

• Board-certified or board-eligible in Internal Medicine or a related subspecialty.

- Current, active, and unrestricted medical license.

• Current, active, and unrestricted medical license.

• Cardiology

• Hematology & Oncology

• Endocrinology

• Nephrology

• Gastroenterology

• Pulmonology

• Other Internal Medicine subspecialties

Preferred Qualifications

• Strong knowledge of evidence-based medicine and clinical practice guidelines.

• Experience in clinical teaching, medical research, or quality improvement initiatives.

• Excellent written communication and analytical skills.

• Interest in artificial intelligence, clinical decision support, or healthcare technology.

Engagement Details

• Fully remote with 100% asynchronous work.

• Standard commitment of approximately 20 hours per week, with opportunities for additional hours based on project demand.

• Flexible working schedule.

• Weekly payments.

• Rolling onboarding with ongoing engagement opportunities.

• Compensation is determined by clinical experience, expertise, and seniority.

Why Join

• Help shape the future of AI-powered clinical decision-making.

• Apply your Internal Medicine expertise to improve advanced healthcare AI systems.

• Collaborate with leading AI researchers and experienced clinicians.

• Contribute to high-impact projects that advance safe, accurate, and evidence-based medical AI.

Equal Opportunity Statement
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.

Contract Information

• Independent contractor engagement.

• Fully remote work completed on your own schedule.

• Projects may be extended, modified, or concluded based on business needs and individual performance.

• Work does not involve access to confidential or proprietary information from any employer, client, or institution.

• Weekly payments are processed through supported payment platforms based on approved work completed.

• Please note that visa sponsorship is not available for this opportunity.