OSCR

From API to Action: A Multi-Model Comparison of OpenAI, Anthropic, Google, and Meta LLMs for Clinical Trial Data Extraction.

Code ↔ Paper

14 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 14 matches
  1. [1] § Appendix A. API Fields Retrieved from ClinicalTrials.gov ↔ notebooks/01_dataset_pull.ipynb, lines 191–265 · score 1.00 · BaselineCategoryTitle, OtherOutcomeDescription, BriefTitle, CompletionDate, ConditionMeshTerm, EligibilityCriteria
  2. [2] § Appendix B. Extraction Prompt and JSON Schema ↔ notebooks/03_LLM_processor.ipynb, lines 547–639 · score 0.95 · brain_stimulation_used, confidence_level, is_noninvasive, primary_type, primary_target, secondary_targets
  3. [3] § 2. Materials and Methods › 2.4. LLM Configuration and Architecture ↔ notebooks/03_LLM_processor.ipynb, lines 166–297 · score 0.79 · max token, meta llama, OpenRouter API, temperature, configured, OpenAI
  4. [4] § 2. Materials and Methods › 2.1. Data Source and Collection ↔ notebooks/01_dataset_pull.ipynb, lines 467–584 · score 0.74 · NCT IDs, eligibility criteria, batch, enrollment, location, status
  5. [5] § 2. Materials and Methods › 2.2. Search Strategy and Trial Selection ↔ figures/make_figure1_search_strategy.py, lines 10–29 · score 0.72 · Parkinson disease tdcs, 19–20, interface, July, variability, clinical
  6. [6] § 2. Materials and Methods › 2.6. Statistical Analysis ↔ notebooks/paper_stats_v2.ipynb, lines 250–341 · score 0.71 · Pearson correlation, Absolute Error, confusion matrices, milliamperes, MAE, numeric
  7. [7] § 2. Materials and Methods › 2.2. Search Strategy and Trial Selection ↔ figures/make_figure1_search_strategy.py, lines 10–29 · score 0.66 · Web interface variability, identical clinical queries, 19–20, Parkinson, July, tDCS
  8. [8] § 2. Materials and Methods › 2.2. Search Strategy and Trial Selection ↔ figures/make_figure1_search_strategy.py, lines 47–61 · score 0.64 · regex filter, tDCS, NCT02349789, NCT07010328, NCT03217110, NCT03221413
  9. [9] § 2. Materials and Methods › 2.6. Statistical Analysis ↔ notebooks/paper_stats_v2.ipynb, lines 106–176 · score 0.63 · motor cortex, tDCS, canonical, transcranial, brain stimulation, class
  10. [10] § 2. Materials and Methods › 2.5. Prompt Design and JSON Schema ↔ notebooks/03_LLM_processor.ipynb, lines 547–639 · score 0.62 · brain regions, relevant quotes, Medium, brain stimulation, noninvasive, confidence
  11. [11] § 3. Results › 3.6. Validation Against an Expert Gold Standard ↔ notebooks/paper_stats_v2.ipynb, lines 106–176 · score 0.61 · primary motor cortex, tDCS, canonical, M1, brain stimulation, synonym
  12. [12] § 3. Results › 3.1. Overall Agreement Patterns ↔ notebooks/paper_stats_table_2.ipynb, lines 176–291 · score 0.57 · intraclass correlation, pairwise comparisons, ICC, rater, intensity, duration
  13. [13] § 2. Materials and Methods › 2.5. Prompt Design and JSON Schema ↔ notebooks/03_LLM_processor.ipynb, lines 919–996 · score 0.56 · BriefSummary, DetailedDescription, brain stimulation, LLM, Prompt, field
  14. [14] § 2. Materials and Methods › 2.6. Statistical Analysis ↔ notebooks/paper_stats_table_2.ipynb, lines 176–291 · score 0.55 · intraclass correlation, ICC, raters, variables, Pairwise, score

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 1,700 lines · 59 KB · MIT · 4 matches

  1. # %%
  2. import os
  3. import pandas as pd
  4. import numpy as np
  5. import openai
  6. import requests
  7. import pandas as pd
  8. import openai
  9. import json
  11. import logging
  12. import time
  14. from typing import List, Dict, Any, Optional
  15. from enum import Enum
  16. from dataclasses import dataclass
  18. import openrouter
  22. # %%
  23. # Import the openrouter module
  25. # Store API key in a separate file
  26. with open('../config/openrouter_api_key.txt', 'r') as f:
  27. api_key = f.read().strip()
  29. openrouter.api_key = api_key
  31. # %%
  32. # Store API key in a separate file
  33. with open('../config/api_key.txt', 'r') as f:
  34. api_key = f.read().strip()
  36. openai.api_key = api_key
  38. # %% [markdown]
  41. # %%
  42. # disease = 'Parkinson'
  44. # %%
  45. # Define file path
  46. file_path_parquet = os.path.join('../data/filtered', '02_simple_filter.parquet')
  48. # Load the DataFrame from the Parquet file
  49. filtered_df_covariates = pd.read_parquet(file_path_parquet)
  51. # Display the DataFrame
  52. print(filtered_df_covariates)
  54. # %%
  55. print(filtered_df_covariates.columns)
  57. # %%
  58. #test openai
  62. # response = openai.chat.completions.create(
  63. # model='gpt-3.5-turbo',
  64. # messages=[
  65. # {"role": "user", "content": "Hello, how are you?"}
  66. # ],
  67. # max_tokens=50,
  68. # temperature=0.5,
  69. # )
  71. # print(response.choices[0].message.content.strip())
  73. # %%
  74. # import requests
  75. # import json
  77. # # Configuration Variables
  78. # API_KEY = openrouter.api_key # Ensure this is defined
  79. # SITE_URL = 'https://pd-research.com' # Your site's URL
  80. # APP_NAME = 'PD Research' # Your application's name
  82. # def send_test_prompt():
  83. # print("Starting the API request...") # Debugging statement
  84. # try:
  85. # # Prepare the payload
  86. # payload = {
  87. # "model": "openai/o1-mini-2024-09-12", # Updated model
  88. # "messages": [
  89. # {
  90. # "role": "user",
  91. # "content": "What is the average age Parkinson's disease starts?"
  92. # }
  93. # ],
  94. # "max_tokens": 150, # Increased max_tokens
  95. # "temperature": 0.7, # Increased temperature for more varied responses
  96. # }
  98. # # Make the POST request to the OpenRouter API
  99. # response = requests.post(
  100. # url="https://openrouter.ai/api/v1/chat/completions",
  101. # headers={
  102. # "Authorization": f"Bearer {API_KEY}",
  103. # "HTTP-Referer": SITE_URL, # Optional: Include if you want your app featured
  104. # "X-Title": APP_NAME, # Optional: Shows your app name on OpenRouter rankings
  105. # },
  106. # data=json.dumps(payload)
  107. # )
  109. # print("API request sent. Awaiting response...") # Debugging statement
  111. # # Check if the request was successful
  112. # if response.status_code == 200:
  113. # print("Received successful response from the API.") # Debugging statement
  114. # response_data = response.json()
  116. # # Print the full JSON response for debugging
  117. # print("Full Response JSON:")
  118. # print(json.dumps(response_data, indent=4))
  120. # # Extract the assistant's reply
  121. # try:
  122. # assistant_reply = response_data['choices'][0]['message']['content'].strip()
  123. # print(f"Assistant's reply: {assistant_reply}")
  124. # except (KeyError, IndexError) as e:
  125. # print("Error extracting assistant's reply:", e)
  126. # print("Full response:", json.dumps(response_data, indent=4))
  127. # else:
  128. # print(f"Request failed with status code {response.status_code}")
  129. # print("Response:", response.text)
  131. # except requests.exceptions.RequestException as e:
  132. # print("An error occurred while making the request:", e)
  134. # # --- Execute the Test Prompt ---
  135. # send_test_prompt()
  137. # %%
  138. def generate_prompt(self, context: str) -> str:
  139. """Generate analysis prompt from context"""
  140. return f"""
  141. Analyze whether brain stimulation was used in this trial. If so, provide details.
  143. IMPORTANT: Respond ONLY with a JSON object in the EXACT format below. Do not include any additional text or explanations.
  145. {{
  146. "brain_stimulation_used": "Yes" or "No",
  147. "stimulation_details": {{
  148. "primary_type": "e.g., tDCS, TMS, tACS, DBS, etc." or null,
  149. "is_noninvasive": true or false,
  150. "primary_target": "Primary brain region or null",
  151. "secondary_targets": ["List of secondary regions"] or [],
  152. "stimulation_parameters": {{
  153. "intensity": "e.g., 2mA" or null,
  154. "duration": "e.g., 20 minutes" or null
  155. }}
  156. }},
  157. "confidence_level": "High", "Medium", or "Low",
  158. "relevant_quotes": ["Direct quotes supporting the analysis"]
  159. }}
  161. Context:
  162. {context}
  163. """
  166. # %%
  167. import openai
  168. import os
  169. import time
  171. from autogen import ConversableAgent, UserProxyAgent
  172. from autogen.agentchat.contrib.capabilities.teachability import Teachability
  174. ###############################################################################
  175. # 1) Hardcode your OpenRouter API key and base URL
  176. ###############################################################################
  177. openrouter_api_key = "sk-or-v1-REPLACE_ME_WITH_REAL_KEY"
  178. openai.api_key = openrouter_api_key
  179. openai.api_base = "https://openrouter.ai/api/v1"
  181. # Optionally set extra OpenRouter headers
  182. openai.request_headers = {
  183. "Authorization": f"Bearer {openrouter_api_key}",
  184. "X-Title": "MyMultiAgentApp",
  185. "HTTP-Referer": "https://example.com",
  186. }
  188. ###############################################################################
  189. # 2) Define multiple model names that are known to work with your key
  190. # (If you don't have GPT-4 access, remove "gpt-4".)
  191. ###############################################################################
  192. model_list = [
  193. "gpt-4",
  194. "anthropic/claude-3.5-sonnet",
  195. "meta-llama/llama-3-70b-instruct",
  196. ]
  198. ###############################################################################
  199. # 3) Build an agent for each model, with Teachability
  200. ###############################################################################
  201. def build_agent_for_model(model_name: str):
  202. """
  203. Creates a ConversableAgent configured for a specific model.
  204. Attaches Teachability with a local vector DB folder
  205. unique to the model_name.
  206. """
  207. config_list = [
  208. {
  209. "model": model_name,
  210. "temperature": 0.5, # or any you prefer
  211. "max_tokens": 4096, # might need a bigger limit for some models
  212. "top_p": 1.0,
  213. }
  214. ]
  216. agent = ConversableAgent(
  217. name=f"agent_{model_name}",
  218. llm_config={
  219. "config_list": config_list,
  220. "timeout": 120,
  221. "cache_seed": None,
  222. },
  223. )
  225. # Use a unique DB path for each model
  226. db_path = f"./tmp/db_{model_name.replace('/','_').replace('-','_')}"
  228. teachability = Teachability(
  229. verbosity=1,
  230. reset_db=True, # Clears any prior memos each run
  231. path_to_db_dir=db_path,
  232. recall_threshold=1.3,
  233. )
  234. teachability.add_to_agent(agent)
  236. return agent
  238. ###############################################################################
  239. # 4) Main Demo
  240. ###############################################################################
  241. def main():
  242. # Hardcoded sports betting text for "teaching"
  243. sports_betting_text = """\
  244. A spread bet in sports betting is a wager based on the margin of victory in a game,
  245. commonly used in popular sports like football and basketball. When you bet on the spread,
  246. you're predicting how much a team will win or lose by.
  248. If you bet on the favorite (shown with a '-' sign), that team must win by more than the
  249. specified point spread for your bet to succeed. If you bet on the underdog (shown with a '+' sign),
  250. they can lose by fewer points than the spread — or win outright — for your bet to win.
  252. The spread helps balance the odds between teams, making the bet more about predicting
  253. the margin rather than just picking a winner.
  255. Example: If the Bears are -6.5 points against the Packers, the Bears must win by more than 6.5
  256. points for a Bears -6.5 bet to cash. If the game ends with the Bears winning by exactly 6 points,
  257. that bet loses. If the underdog (Packers) loses by fewer than 6.5, an underdog bet would win.
  258. """
  260. # Create one user proxy. We'll call user.initiate_chat(...) ourselves.
  261. user = UserProxyAgent(
  262. name="user",
  263. human_input_mode="NEVER", # We handle input() in Python code
  264. max_consecutive_auto_reply=0,
  265. )
  267. # Build an agent for each model
  268. agents = {}
  269. for m in model_list:
  270. agents[m] = build_agent_for_model(m)
  272. # 4a) Teach each agent the sports-betting text once
  273. for model_name, agent in agents.items():
  274. print(f"\n>>> Teaching {model_name} the sports-betting text...\n")
  275. teach_resp = user.initiate_chat(agent, message=sports_betting_text, clear_history=True)
  276. print(f"{model_name} replied:", teach_resp.content)
  278. # 4b) Now we do a Q&A loop: user types question, all agents respond
  279. print("\n=== Multi-Model Q&A ===")
  280. print("Type 'exit' to quit.\n")
  281. while True:
  282. q = input("You: ").strip()
  283. if q.lower() in ["exit", "quit"]:
  284. print("Goodbye!")
  285. break
  287. for model_name, agent in agents.items():
  288. # Start a new chat to confirm it can recall the taught text
  289. # Or you can do clear_history=False so it accumulates Q&A
  290. answer = user.initiate_chat(agent, message=q, clear_history=True)
  291. print(f"\n[{model_name} responds]\n{answer.content}\n")
  293. # Optional: small delay if you want to separate them visually
  294. time.sleep(1)
  296. if __name__ == "__main__":
  297. main()
  299. # %%
  300. import requests
  304. resp = requests.post('https://textbelt.com/text', {
  306. 'phone': '9163802941',
  308. 'message': 'PD Pipe Line Part 3 Done',
  310. 'key': '138adc496234ca311154757db147f552afa8ba83FfrCKJ36kTJNXq65nlsvvF4Pu',
  312. })
  314. print(resp.json())
  316. # %%
  319. 1=2
  322. # %%
  325. # %%
  326. # old working code
  328. # %%
  329. import os
  330. import requests
  331. import json
  332. import logging
  333. import pandas as pd
  334. import time
  335. from typing import Dict, List, Optional, Union
  336. from dataclasses import dataclass
  337. from enum import Enum
  339. # Configure logging
  340. logging.basicConfig(
  341. level=logging.INFO,
  342. format='%(asctime)s - %(levelname)s - %(message)s',
  343. datefmt='%Y-%m-%d %H:%M:%S'
  344. )
  346. class APIError(Exception):
  347. """Custom exception for API-related errors"""
  348. pass
  350. class ValidationError(Exception):
  351. """Custom exception for response validation errors"""
  352. pass
  354. class BrainStimStatus(str, Enum):
  355. YES = "Yes"
  356. NO = "No"
  358. @dataclass
  359. class StimulationParameters:
  360. frequency: Optional[str] = None
  361. intensity: Optional[str] = None
  362. duration: Optional[str] = None
  364. @dataclass
  365. class StimulationDetails:
  366. primary_type: Optional[str] = None
  367. is_noninvasive: Optional[bool] = None
  368. primary_target: Optional[str] = None
  369. secondary_targets: List[str] = None
  370. stimulation_parameters: StimulationParameters = None
  372. def __post_init__(self):
  373. if self.secondary_targets is None:
  374. self.secondary_targets = []
  375. if self.stimulation_parameters is None:
  376. self.stimulation_parameters = StimulationParameters()
  378. @dataclass
  379. class BrainStimResponse:
  380. brain_stimulation_used: BrainStimStatus
  381. stimulation_details: StimulationDetails
  382. confidence_level: str
  383. relevant_quotes: List[str]
  385. class BrainStimAnalyzer:
  386. def __init__(self, api_key: str, site_url: str, app_name: str):
  387. self.api_key = api_key
  388. self.site_url = site_url
  389. self.app_name = app_name
  390. self.models = {
  391. "gpt-4": {
  392. "requires_image": False,
  393. "max_tokens": 4096,
  394. "temperature": 0.5,
  395. "instructions": "Return ONLY a JSON object with no additional explanatory text."
  396. },
  397. "anthropic/claude-3.5-sonnet": {
  398. "requires_image": False,
  399. "max_tokens": 10000,
  400. "temperature": 0.5,
  401. "instructions": "Return ONLY a JSON object with no additional explanatory text."
  402. },
  403. "meta-llama/llama-3-70b-instruct": {
  404. "requires_image": False,
  405. "max_tokens": 4096,
  406. "temperature": 0.5,
  407. "instructions": "Return ONLY a JSON object with no additional explanatory text."
  408. }
  409. }
  411. def extract_json_from_text(self, text: str) -> Optional[str]:
  412. """Extract JSON object from text that may contain additional content"""
  413. try:
  414. # Look for JSON-like content between curly braces
  415. start = text.find('{')
  416. end = text.rfind('}')
  417. if start != -1 and end != -1:
  418. json_str = text[start:end+1]
  419. # Validate it's proper JSON
  420. json.loads(json_str)
  421. return json_str
  422. except json.JSONDecodeError:
  423. pass
  424. return None
  426. def validate_brain_stim_response(self, response_json: Dict) -> BrainStimResponse:
  427. """Validate response structure and content"""
  428. required_fields = [
  429. "brain_stimulation_used",
  430. "stimulation_details",
  431. "confidence_level",
  432. "relevant_quotes"
  433. ]
  435. # Check required fields
  436. for field in required_fields:
  437. if field not in response_json:
  438. raise ValidationError(f"Missing required field: {field}")
  440. # Validate brain_stimulation_used
  441. if not isinstance(response_json["brain_stimulation_used"], str):
  442. raise ValidationError("brain_stimulation_used must be string")
  443. if response_json["brain_stimulation_used"] not in ["Yes", "No"]:
  444. raise ValidationError("brain_stimulation_used must be 'Yes' or 'No'")
  446. # Convert to BrainStimResponse object
  447. stim_params = StimulationParameters(**response_json["stimulation_details"]["stimulation_parameters"])
  448. stim_details = StimulationDetails(
  449. primary_type=response_json["stimulation_details"]["primary_type"],
  450. is_noninvasive=response_json["stimulation_details"]["is_noninvasive"],
  451. primary_target=response_json["stimulation_details"]["primary_target"],
  452. secondary_targets=response_json["stimulation_details"]["secondary_targets"],
  453. stimulation_parameters=stim_params
  454. )
  456. return BrainStimResponse(
  457. brain_stimulation_used=BrainStimStatus(response_json["brain_stimulation_used"]),
  458. stimulation_details=stim_details,
  459. confidence_level=response_json["confidence_level"],
  460. relevant_quotes=response_json["relevant_quotes"]
  461. )
  463. def send_request(self, model_name: str, prompt: str, max_retries: int = 3) -> BrainStimResponse:
  464. """Send request to API with retry logic"""
  465. if model_name not in self.models:
  466. raise ValueError(f"Unknown model: {model_name}")
  468. model_config = self.models[model_name]
  469. prompt = f"{model_config['instructions']}\n{prompt}"
  471. retries = 0
  472. backoff = 2 # seconds
  474. while retries < max_retries:
  475. try:
  476. response = requests.post(
  477. url="https://openrouter.ai/api/v1/chat/completions",
  478. headers={
  479. "Authorization": f"Bearer {self.api_key}",
  480. "HTTP-Referer": self.site_url,
  481. "X-Title": self.app_name,
  482. "Content-Type": "application/json"
  483. },
  484. json={
  485. "model": model_name,
  486. "messages": [{"role": "user", "content": prompt}],
  487. "temperature": model_config["temperature"],
  488. "max_tokens": model_config["max_tokens"]
  489. },
  490. timeout=60
  491. )
  493. logging.info(f"Sent request to {model_name}")
  494. logging.debug(f"Raw response: {response.text}")
  496. if response.status_code == 200:
  497. try:
  498. response_data = response.json()
  499. # Check if 'choices' and necessary keys exist in the response
  500. if 'choices' in response_data and len(response_data['choices']) > 0:
  501. choice = response_data['choices'][0]
  502. if 'message' in choice and 'content' in choice['message']:
  503. content = choice['message']['content'].strip()
  505. # Try to parse as JSON directly first
  506. try:
  507. response_json = json.loads(content)
  508. except json.JSONDecodeError:
  509. # If direct parsing fails, try to extract JSON from text
  510. json_str = self.extract_json_from_text(content)
  511. if not json_str:
  512. raise ValidationError(f"Could not extract valid JSON from response: {content}")
  513. response_json = json.loads(json_str)
  515. # Validate response structure and content
  516. return self.validate_brain_stim_response(response_json)
  517. else:
  518. raise ValidationError("Response does not contain 'message' or 'content' keys.")
  519. else:
  520. raise ValidationError("Response does not contain 'choices' or it's empty.")
  522. except (KeyError, IndexError) as e:
  523. logging.error(f"Error extracting response content: {e}")
  524. raise ValidationError(f"Invalid response structure from {model_name}")
  526. elif response.status_code in [429, 500, 502, 503, 504]:
  527. if retries == max_retries - 1:
  528. raise APIError(f"Max retries reached for {model_name}")
  529. retries += 1
  530. time.sleep(backoff)
  531. backoff *= 2
  532. continue
  534. else:
  535. raise APIError(f"Request failed with status {response.status_code}: {response.text}")
  537. except requests.exceptions.RequestException as e:
  538. if retries == max_retries - 1:
  539. raise APIError(f"Request failed after {max_retries} retries: {str(e)}")
  540. retries += 1
  541. time.sleep(backoff)
  542. backoff *= 2
  543. continue
  545. raise APIError(f"Failed to get valid response from {model_name}")
  547. class BrainStimAnalysis:
  548. def __init__(self, api_key: str, site_url: str, app_name: str):
  549. self.analyzer = BrainStimAnalyzer(api_key, site_url, app_name)
  551. def process_trial(self, df: pd.DataFrame, output_dir: str = '03_data_ai'):
  552. """Process trial data and save results"""
  553. os.makedirs(output_dir, exist_ok=True)
  554. result_df = df.copy()
  556. # Initialize response arrays
  557. responses = []
  559. # Process each row
  560. for idx, row in df.iterrows():
  561. context = row.get('DetailedDescription', row.get('BriefSummary', ''))
  562. prompt = self.generate_prompt(context)
  564. # Get responses from each model
  565. for model_name in self.analyzer.models.keys():
  566. try:
  567. response = self.analyzer.send_request(model_name, prompt)
  568. responses.append({
  569. "Model": model_name,
  570. "Response": response
  571. })
  573. # Update DataFrame with response data
  574. self.update_dataframe(result_df, idx, model_name, response)
  576. except (APIError, ValidationError) as e:
  577. logging.error(f"Error processing row {idx} with model {model_name}: {str(e)}")
  578. continue
  580. # Save results
  581. self.save_results(result_df, output_dir)
  582. return result_df, responses
  584. def generate_prompt(self, context: str) -> str:
  585. """Generate analysis prompt from context"""
  586. return f"""
  587. Analyze whether brain stimulation was used in this trial. If so, provide details.
  589. IMPORTANT: Respond ONLY with a JSON object in the EXACT format below. Do not include any additional text or explanations.
  591. {{
  592. "brain_stimulation_used": "Yes" or "No",
  593. "stimulation_details": {{
  594. "primary_type": "e.g., tDCS, TMS, tACS, DBS, etc." or null,
  595. "is_noninvasive": true or false,
  596. "primary_target": "Primary brain region or null",
  597. "secondary_targets": ["List of secondary regions"] or [],
  598. "stimulation_parameters": {{
  599. "frequency": "e.g., 10Hz" or null,
  600. "intensity": "e.g., 2mA" or null,
  601. "duration": "e.g., 20 minutes" or null
  602. }}
  603. }},
  604. "confidence_level": "High", "Medium", or "Low",
  605. "relevant_quotes": ["Direct quotes supporting the analysis"]
  606. }}
  608. Context:
  609. {context}
  610. """
  612. def update_dataframe(self, df: pd.DataFrame, idx: int, model_name: str, response: BrainStimResponse):
  613. """Update DataFrame with response data"""
  614. suffix = model_name.replace("/", "_").replace("-", "_")
  616. # Update brain stimulation status
  617. df.at[idx, f"brain_stimulation_used_{suffix}"] = response.brain_stimulation_used.value
  619. # Update stimulation details
  620. if response.stimulation_details:
  621. df.at[idx, f"stimulation_details_primary_type_{suffix}"] = response.stimulation_details.primary_type
  622. df.at[idx, f"stimulation_details_is_noninvasive_{suffix}"] = response.stimulation_details.is_noninvasive
  623. df.at[idx, f"stimulation_details_primary_target_{suffix}"] = response.stimulation_details.primary_target
  624. df.at[idx, f"stimulation_details_secondary_targets_{suffix}"] = json.dumps(response.stimulation_details.secondary_targets)
  626. # Update parameters
  627. params = response.stimulation_details.stimulation_parameters
  628. df.at[idx, f"stimulation_details_parameters_frequency_{suffix}"] = params.frequency
  629. df.at[idx, f"stimulation_details_parameters_intensity_{suffix}"] = params.intensity
  630. df.at[idx, f"stimulation_details_parameters_duration_{suffix}"] = params.duration
  632. # Update confidence and quotes
  633. df.at[idx, f"confidence_level_{suffix}"] = response.confidence_level
  634. df.at[idx, f"relevant_quotes_{suffix}"] = json.dumps(response.relevant_quotes)
  636. def save_results(self, df: pd.DataFrame, output_dir: str):
  637. """Save results to files"""
  638. df.to_excel(f'{output_dir}/part_3_covariates_done.xlsx', index=False)
  639. df.to_parquet(f'{output_dir}/part_3_covariates_done.parquet', index=False)
  641. def main():
  642. # Configuration
  643. api_key = openrouter.api_key
  644. site_url = 'https://pd-research.com'
  645. app_name = 'PD Research'
  647. if not api_key:
  648. logging.error("API_KEY is not set. Please set the OPENROUTER_API_KEY environment variable.")
  649. exit(1)
  651. try:
  652. # Load your DataFrame (replace with actual loading logic)
  653. df = filtered_df_covariates ## Update with your file path
  655. # Initialize and run analysis
  656. analysis = BrainStimAnalysis(api_key, site_url, app_name)
  657. result_df, responses = analysis.process_trial(df)
  660. # Save result_df and responses to Excel and Parquet formats
  661. output_dir = '03_data_ai'
  662. os.makedirs(output_dir, exist_ok=True)
  664. # Save result_df
  665. result_df.to_excel(f'{output_dir}/part_3_covariates_done_v2.xlsx', index=False)
  666. result_df.to_parquet(f'{output_dir}/part_3_covariates_done_v2.parquet', index=False)
  668. # Convert responses to DataFrame and save
  669. responses_df = pd.DataFrame(responses)
  670. responses_df.to_excel(f'{output_dir}/responses_done_v2.xlsx', index=False)
  671. responses_df.to_parquet(f'{output_dir}/responses_done_v2.parquet', index=False)
  673. logging.info(f"Results saved to {output_dir}/part_3_covariates_done_v2.xlsx and {output_dir}/responses_done_v2.xlsx")
  677. # Display results
  678. print("\n=== Updated LLM Responses ===")
  679. print(result_df)
  681. # Optional: Display using PrettyTable
  682. try:
  683. from prettytable import PrettyTable
  684. table = PrettyTable()
  685. table.field_names = ["Model", "Response"]
  686. for resp in responses:
  687. table.add_row([resp["Model"], resp["Response"]])
  688. print("\n=== LLM Responses (PrettyTable) ===")
  689. print(table)
  690. except ImportError:
  691. logging.warning("PrettyTable not installed. Skipping table formatting.")
  693. except Exception as e:
  694. logging.error(f"Error in main execution: {str(e)}")
  695. raise
  700. # first_five.to_excel('filtered_df_covariates_updated.xlsx', index=False)
  701. # logging.info("Updated DataFrame has been saved to filtered_df_covariates_updated.xlsx")
  704. if __name__ == "__main__":
  705. main()
  707. # %% [markdown]
  708. # old code working
  710. # %%
  711. # Save the updated DataFrame to files in folder '03_data_ai'
  712. responses.to_excel('03_data_ai/part_3_covariates_done_v2.xlsx', index=False)
  714. # result_df.to_parquet('03_data_ai/part_3_covariates_done_v2.parquet', index=False)
  718. # %%
  719. 1=2
  722. # %%
  723. import os
  724. import requests
  725. import json
  726. import logging
  727. import pandas as pd
  728. import time
  730. # Configure logging
  731. logging.basicConfig(level=logging.INFO, format='%(levelname)s:%(message)s')
  733. # Configuration Variables
  734. API_KEY = openrouter.api_key
  735. if not API_KEY:
  736. logging.error("API_KEY is not set. Please set the OPENROUTER_API_KEY environment variable.")
  737. exit(1)
  739. SITE_URL = 'https://pd-research.com' # Your site's URL
  740. APP_NAME = 'PD Research' # Your application's name
  742. # Define the models to query with their specific settings
  743. MODELS = {
  744. "gpt-4": {
  745. "requires_image": False,
  746. "payload_modifier": lambda content: content, # No modification needed
  747. "max_tokens": 4096, # Example max_tokens for GPT-4
  748. "temperature": 0.5 # Lower temperature for accuracy
  749. },
  750. "anthropic/claude-3.5-sonnet": {
  751. "requires_image": False, # Changed to False as per your requirement
  752. "payload_modifier": lambda content: content, # No modification needed
  753. "max_tokens": 10000, # Example max_tokens for Claude 3.5
  754. "temperature": 0.5 # Lower temperature for accuracy
  755. },
  756. "meta-llama/llama-3-70b-instruct": {
  757. "requires_image": False,
  758. "payload_modifier": lambda content: content, # No modification needed
  759. "max_tokens": 4096, # Example max_tokens for Qwen
  760. "temperature": 0.5 # Lower temperature for accuracy
  761. }
  762. }
  764. # Define colors for each model (Optional: Requires colorama)
  765. try:
  766. from colorama import Fore, Style
  767. COLORAMA_AVAILABLE = True
  768. except ImportError:
  769. COLORAMA_AVAILABLE = False
  770. logging.warning("Colorama not installed. Responses will not be color-coded.")
  772. MODEL_COLORS = {
  773. "gpt-4": Fore.BLUE if COLORAMA_AVAILABLE else "",
  774. "anthropic/claude-3.5-sonnet": Fore.MAGENTA if COLORAMA_AVAILABLE else "",
  775. "meta-llama/llama-3-70b-instruct": Fore.GREEN if COLORAMA_AVAILABLE else ""
  776. }
  778. def send_request(model_name, prompt):
  779. """
  780. Sends a request to the specified model with the given prompt.
  782. Parameters:
  783. model_name (str): The name of the model to query.
  784. prompt (str): The text prompt/question.
  786. Returns:
  787. str: The assistant's reply or an error message.
  788. """
  789. logging.info(f"Sending request to model: {model_name}")
  791. # For text-only models, no modification is needed
  792. content = prompt
  793. modified_content = MODELS[model_name]["payload_modifier"](content)
  795. # Construct the payload with per-model settings
  796. payload = {
  797. "model": model_name,
  798. "messages": [
  799. {
  800. "role": "user",
  801. "content": modified_content
  802. }
  803. ],
  804. "top_p": 1,
  805. "temperature": MODELS[model_name]["temperature"], # Use per-model temperature
  806. "frequency_penalty": 0,
  807. "presence_penalty": 0,
  808. "repetition_penalty": 1,
  809. "top_k": 0,
  810. "max_tokens": MODELS[model_name]["max_tokens"], # Use per-model max_tokens
  811. }
  813. retries = 3
  814. backoff = 2 # seconds
  816. for attempt in range(1, retries + 1):
  817. try:
  818. # Make the POST request to the OpenRouter API
  819. response = requests.post(
  820. url="https://openrouter.ai/api/v1/chat/completions",
  821. headers={
  822. "Authorization": f"Bearer {API_KEY}",
  823. "HTTP-Referer": SITE_URL, # Optional: Include if you want your app featured
  824. "X-Title": APP_NAME, # Optional: Shows your app name on OpenRouter rankings
  825. "Content-Type": "application/json" # Ensure content type is set
  826. },
  827. data=json.dumps(payload),
  828. timeout=60 # Increased timeout for models that may take longer
  829. )
  831. logging.info("API request sent. Awaiting response...")
  833. # Check Content-Type
  834. content_type = response.headers.get('Content-Type', '')
  835. logging.debug(f"Response Content-Type: {content_type}")
  836. if 'application/json' not in content_type:
  837. logging.error(f"Unexpected Content-Type: {content_type}")
  838. logging.error(f"Response: {response.text}")
  839. return "Error: Unexpected Content-Type"
  841. # Check if the request was successful
  842. if response.status_code == 200:
  843. logging.info(f"Received successful response from {model_name}.")
  844. try:
  845. response_data = response.json()
  846. # Extract the assistant's reply
  847. assistant_reply = response_data['choices'][0]['message']['content'].strip()
  848. logging.info(f"Assistant's reply from {model_name}: {assistant_reply}")
  849. return assistant_reply
  850. except (KeyError, IndexError) as e:
  851. logging.error(f"Error extracting assistant's reply from {model_name}: {e}")
  852. logging.debug(f"Full response from {model_name}: {json.dumps(response_data, indent=4)}")
  853. return "Error extracting reply."
  854. elif response.status_code in [429, 500, 502, 503, 504]:
  855. # Retry for rate limiting or server errors
  856. logging.warning(f"Attempt {attempt} failed with status code {response.status_code}. Retrying in {backoff} seconds...")
  857. time.sleep(backoff)
  858. backoff *= 2
  859. else:
  860. logging.error(f"Request to {model_name} failed with status code {response.status_code}")
  861. logging.error(f"Response: {response.text}")
  862. return f"Error: Received status code {response.status_code}"
  863. except requests.exceptions.RequestException as e:
  864. logging.error(f"Attempt {attempt} failed with exception: {e}")
  865. if attempt < retries:
  866. logging.info(f"Retrying in {backoff} seconds...")
  867. time.sleep(backoff)
  868. backoff *= 2
  869. else:
  870. logging.error("All retry attempts failed.")
  871. return f"Error: {e}"
  873. return "Error: Unable to retrieve response."
  875. def main():
  876. # Load your DataFrame
  877. # Replace the following line with how you load your DataFrame
  878. # For example, if it's a CSV:
  879. # filtered_df_covariates = pd.read_csv('filtered_df_covariates.csv')
  881. # For demonstration, let's assume filtered_df_covariates is already loaded
  882. # Here's a placeholder:
  883. # filtered_df_covariates = pd.DataFrame(...) # Your DataFrame loading logic
  885. # **Important:** Replace the following line with your actual DataFrame loading method
  886. # For example:
  887. # filtered_df_covariates = pd.read_csv('your_file.csv')
  888. # Ensure that 'filtered_df_covariates' is defined before proceeding
  889. try:
  890. filtered_df_covariates # Check if the DataFrame is already loaded
  891. except NameError:
  892. logging.error("DataFrame 'filtered_df_covariates' is not defined. Please load it before running the script.")
  893. exit(1)
  895. # Define the prompt
  896. prompt_template = """
  897. Analyze whether brain stimulation was used in this trial. If so, provide details.
  899. IMPORTANT: Respond ONLY with a JSON object in the EXACT format below. Do not include any additional text or explanations.
  901. {
  902. "brain_stimulation_used": "Yes" or "No",
  903. "stimulation_details": {
  904. "primary_type": "e.g., tDCS, TMS, tACS, DBS, etc." or null,
  905. "is_noninvasive": true or false,
  906. "primary_target": "Primary brain region or null",
  907. "secondary_targets": ["List of secondary regions"] or [],
  908. "stimulation_parameters": {
  909. "frequency": "e.g., 10Hz" or null,
  910. "intensity": "e.g., 2mA" or null,
  911. "duration": "e.g., 20 minutes" or null
  912. }
  913. },
  914. "confidence_level": "High", "Medium", or "Low",
  915. "relevant_quotes": ["Direct quotes supporting the analysis"]
  916. }
  917. """
  919. # Select the first 5 rows
  920. first_five = filtered_df_covariates.copy()
  922. # Define new columns for JSON fields with model suffixes
  923. # We'll create columns dynamically as we process each model
  924. # Alternatively, initialize them if you know the models beforehand
  926. # Initialize the responses list
  927. responses = []
  929. # Iterate over the first 5 rows
  930. for idx, row in first_five.iterrows():
  931. # Extract relevant information from the row to include in the prompt
  932. # Customize this based on which columns provide context for the analysis
  933. # For example, using 'DetailedDescription' or 'FullAnalysis'
  934. context = row['DetailedDescription'] if 'DetailedDescription' in row else row['BriefSummary']
  936. # Construct the full prompt by appending context to the prompt template
  937. prompt = f"{prompt_template}\n\nContext:\n{context}"
  939. # Send the prompt to each model and store responses
  940. for model in MODELS.keys():
  941. response = send_request(model, prompt)
  942. responses.append({
  943. "Model": model,
  944. "Response": response
  945. })
  947. # Check if the response is a JSON object
  948. try:
  949. response_json = json.loads(response)
  950. except json.JSONDecodeError:
  951. logging.error(f"Response from {model} is not valid JSON. Response: {response}")
  952. response_json = {}
  954. # Define column suffix based on model name
  955. suffix = model.replace("/", "_").replace("-", "_")
  957. # Populate the new columns with the JSON data
  958. if "brain_stimulation_used" in response_json:
  959. first_five.at[idx, f"brain_stimulation_used_{suffix}"] = response_json.get("brain_stimulation_used", None)
  960. else:
  961. first_five.at[idx, f"brain_stimulation_used_{suffix}"] = None
  963. if "stimulation_details" in response_json and isinstance(response_json["stimulation_details"], dict):
  964. stim_details = response_json["stimulation_details"]
  965. first_five.at[idx, f"stimulation_details_primary_type_{suffix}"] = stim_details.get("primary_type", None)
  966. first_five.at[idx, f"stimulation_details_is_noninvasive_{suffix}"] = stim_details.get("is_noninvasive", None)
  967. first_five.at[idx, f"stimulation_details_primary_target_{suffix}"] = stim_details.get("primary_target", None)
  968. # Assign as JSON string to avoid issues with lists
  969. secondary_targets = stim_details.get("secondary_targets", [])
  970. first_five.at[idx, f"stimulation_details_secondary_targets_{suffix}"] = json.dumps(secondary_targets)
  972. stim_params = stim_details.get("stimulation_parameters", {})
  973. first_five.at[idx, f"stimulation_details_stimulation_parameters_frequency_{suffix}"] = stim_params.get("frequency", None)
  974. first_five.at[idx, f"stimulation_details_stimulation_parameters_intensity_{suffix}"] = stim_params.get("intensity", None)
  975. first_five.at[idx, f"stimulation_details_stimulation_parameters_duration_{suffix}"] = stim_params.get("duration", None)
  976. else:
  977. # If 'stimulation_details' is missing or not a dict, set defaults
  978. first_five.at[idx, f"stimulation_details_primary_type_{suffix}"] = None
  979. first_five.at[idx, f"stimulation_details_is_noninvasive_{suffix}"] = None
  980. first_five.at[idx, f"stimulation_details_primary_target_{suffix}"] = None
  981. first_five.at[idx, f"stimulation_details_secondary_targets_{suffix}"] = json.dumps([])
  982. first_five.at[idx, f"stimulation_details_stimulation_parameters_frequency_{suffix}"] = None
  983. first_five.at[idx, f"stimulation_details_stimulation_parameters_intensity_{suffix}"] = None
  984. first_five.at[idx, f"stimulation_details_stimulation_parameters_duration_{suffix}"] = None
  986. if "confidence_level" in response_json:
  987. first_five.at[idx, f"confidence_level_{suffix}"] = response_json.get("confidence_level", None)
  988. else:
  989. first_five.at[idx, f"confidence_level_{suffix}"] = None
  991. if "relevant_quotes" in response_json:
  992. # Assign as JSON string to avoid issues with lists
  993. relevant_quotes = response_json.get("relevant_quotes", [])
  994. first_five.at[idx, f"relevant_quotes_{suffix}"] = json.dumps(relevant_quotes)
  995. else:
  996. first_five.at[idx, f"relevant_quotes_{suffix}"] = json.dumps([])
  998. # Display the updated DataFrame
  999. print("\n=== Updated LLM Responses ===")
  1000. # Select only the new columns for display
  1001. new_columns_display = []
  1002. for model in MODELS.keys():
  1003. suffix = model.replace("/", "_").replace("-", "_")
  1004. new_columns_display.extend([
  1005. f"brain_stimulation_used_{suffix}",
  1006. f"stimulation_details_primary_type_{suffix}",
  1007. f"stimulation_details_is_noninvasive_{suffix}",
  1008. f"stimulation_details_primary_target_{suffix}",
  1009. f"stimulation_details_secondary_targets_{suffix}",
  1010. f"stimulation_details_stimulation_parameters_frequency_{suffix}",
  1011. f"stimulation_details_stimulation_parameters_intensity_{suffix}",
  1012. f"stimulation_details_stimulation_parameters_duration_{suffix}",
  1013. f"confidence_level_{suffix}",
  1014. f"relevant_quotes_{suffix}"
  1015. ])
  1017. print(first_five[new_columns_display])
  1019. # Optional: Using PrettyTable for better console display
  1020. try:
  1021. from prettytable import PrettyTable
  1022. table = PrettyTable()
  1023. table.field_names = ["Model", "Response"]
  1025. for resp in responses:
  1026. table.add_row([resp["Model"], resp["Response"]])
  1028. print("\n=== LLM Responses (PrettyTable) ===")
  1029. print(table)
  1030. except ImportError:
  1031. logging.warning("PrettyTable not installed. Skipping table formatting.")
  1033. # Optional: Using colorama for color-coded console output
  1034. if COLORAMA_AVAILABLE:
  1035. print("\n=== LLM Responses (Color-Coded) ===")
  1036. for resp in responses:
  1037. color = MODEL_COLORS.get(resp["Model"], Fore.WHITE)
  1038. print(f"{color}{resp['Model']}: {resp['Response']}{Style.RESET_ALL}")
  1039. else:
  1040. logging.info("Colorama not available. Skipping color-coded output.")
  1042. # Optional: Save the updated DataFrame to a file
  1043. # Uncomment the following lines to save as CSV or Excel
  1044. # first_five.to_csv('filtered_df_covariates_updated.csv', index=False)
  1045. # Ensure the new columns are in the DataFrame
  1046. for model in MODELS.keys():
  1047. suffix = model.replace("/", "_").replace("-", "_")
  1048. new_columns = [
  1049. f"brain_stimulation_used_{suffix}",
  1050. f"stimulation_details_primary_type_{suffix}",
  1051. f"stimulation_details_is_noninvasive_{suffix}",
  1052. f"stimulation_details_primary_target_{suffix}",
  1053. f"stimulation_details_secondary_targets_{suffix}",
  1054. f"stimulation_details_stimulation_parameters_frequency_{suffix}",
  1055. f"stimulation_details_stimulation_parameters_intensity_{suffix}",
  1056. f"stimulation_details_stimulation_parameters_duration_{suffix}",
  1057. f"confidence_level_{suffix}",
  1058. f"relevant_quotes_{suffix}"
  1059. ]
  1060. for col in new_columns:
  1061. if col not in first_five.columns:
  1062. first_five[col] = None
  1064. # Ensure the directory '03_data_ai' exists
  1065. os.makedirs('03_data_ai', exist_ok=True)
  1067. # Save the updated DataFrame to files in folder '03_data_ai'
  1068. first_five.to_excel('03_data_ai/part_3_covariates_done.xlsx', index=False)
  1069. first_five.to_parquet('03_data_ai/part_3_covariates_done.parquet', index=False)
  1072. # first_five.to_excel('filtered_df_covariates_updated.xlsx', index=False)
  1073. # logging.info("Updated DataFrame has been saved to filtered_df_covariates_updated.xlsx")
  1075. if __name__ == "__main__":
  1076. main()
  1078. # %%
  1079. # # Save the DataFrame to a CSV file
  1080. # filtered_df_covariates.to_csv('03_data_ai/part_3_covariates_done.csv', index=False)
  1081. # filtered_df_covariates.to_excel('03_data_ai/part_3_covariates_done.xlsx', index=False)
  1082. # filtered_df_covariates.to_parquet('03_data_ai/part_3_covariates_done.parquet', index=False)
  1084. # %%
  1085. # Save the DataFrame to an Excel file
  1087. # %%
  1088. 1=2
  1091. # %%
  1092. # WORKING - BELOW÷
  1094. # %%
  1095. # import os
  1096. # import requests
  1097. # import json
  1098. # import logging
  1099. # import pandas as pd
  1100. # import time
  1102. # # Configure logging
  1103. # # logging.basicConfig(level=logging.INFO, format='%(levelname)s:%(message)s')
  1106. # SITE_URL = 'https://pd-research.com' # Your site's URL
  1107. # APP_NAME = 'PD Research' # Your application's name
  1109. # # Define the models to query with their specific settings
  1110. # MODELS = {
  1111. # "gpt-4": {
  1112. # "requires_image": False,
  1113. # "payload_modifier": lambda content: content, # No modification needed
  1114. # "max_tokens": 4096, # Example max_tokens for GPT-4
  1115. # "temperature": 0.5 # Lower temperature for accuracy
  1116. # },
  1117. # "anthropic/claude-3.5-sonnet": {
  1118. # "requires_image": False, # Changed to False as per your requirement
  1119. # "payload_modifier": lambda content: content, # No modification needed
  1120. # "max_tokens": 10000, # Example max_tokens for Claude 3.5
  1121. # "temperature": 0.5 # Lower temperature for accuracy
  1122. # },
  1123. # "qwen/qwen-2.5-72b-instruct": {
  1124. # "requires_image": False,
  1125. # "payload_modifier": lambda content: content, # No modification needed
  1126. # "max_tokens": 4096, # Example max_tokens for Qwen
  1127. # "temperature": 0.5 # Lower temperature for accuracy
  1128. # }
  1129. # }
  1131. # # Define colors for each model (Optional: Requires colorama)
  1132. # try:
  1133. # from colorama import Fore, Style
  1134. # COLORAMA_AVAILABLE = True
  1135. # except ImportError:
  1136. # COLORAMA_AVAILABLE = False
  1137. # logging.warning("Colorama not installed. Responses will not be color-coded.")
  1139. # MODEL_COLORS = {
  1140. # "gpt-4": Fore.BLUE if COLORAMA_AVAILABLE else "",
  1141. # "anthropic/claude-3.5-sonnet": Fore.MAGENTA if COLORAMA_AVAILABLE else "",
  1142. # "qwen/qwen-2.5-72b-instruct": Fore.GREEN if COLORAMA_AVAILABLE else ""
  1143. # }
  1145. # def send_request(model_name, prompt):
  1146. # """
  1147. # Sends a request to the specified model with the given prompt.
  1149. # Parameters:
  1150. # model_name (str): The name of the model to query.
  1151. # prompt (str): The text prompt/question.
  1153. # Returns:
  1154. # str: The assistant's reply or an error message.
  1155. # """
  1156. # logging.info(f"Sending request to model: {model_name}")
  1158. # # For text-only models, no modification is needed
  1159. # content = prompt
  1160. # modified_content = MODELS[model_name]["payload_modifier"](content)
  1162. # # Construct the payload with per-model settings
  1163. # payload = {
  1164. # "model": model_name,
  1165. # "messages": [
  1166. # {
  1167. # "role": "user",
  1168. # "content": modified_content
  1169. # }
  1170. # ],
  1171. # "top_p": 1,
  1172. # "temperature": MODELS[model_name]["temperature"], # Use per-model temperature
  1173. # "frequency_penalty": 0,
  1174. # "presence_penalty": 0,
  1175. # "repetition_penalty": 1,
  1176. # "top_k": 0,
  1177. # "max_tokens": MODELS[model_name]["max_tokens"], # Use per-model max_tokens
  1178. # }
  1180. # retries = 3
  1181. # backoff = 2 # seconds
  1183. # for attempt in range(1, retries + 1):
  1184. # try:
  1185. # # Make the POST request to the OpenRouter API
  1186. # response = requests.post(
  1187. # url="https://openrouter.ai/api/v1/chat/completions",
  1188. # headers={
  1189. # "Authorization": f"Bearer {API_KEY}",
  1190. # "HTTP-Referer": SITE_URL, # Optional: Include if you want your app featured
  1191. # "X-Title": APP_NAME, # Optional: Shows your app name on OpenRouter rankings
  1192. # "Content-Type": "application/json" # Ensure content type is set
  1193. # },
  1194. # data=json.dumps(payload),
  1195. # timeout=60 # Increased timeout for models that may take longer
  1196. # )
  1198. # logging.info("API request sent. Awaiting response...")
  1200. # # Check Content-Type
  1201. # content_type = response.headers.get('Content-Type', '')
  1202. # logging.debug(f"Response Content-Type: {content_type}")
  1203. # if 'application/json' not in content_type:
  1204. # logging.error(f"Unexpected Content-Type: {content_type}")
  1205. # logging.error(f"Response: {response.text}")
  1206. # return "Error: Unexpected Content-Type"
  1208. # # Check if the request was successful
  1209. # if response.status_code == 200:
  1210. # logging.info(f"Received successful response from {model_name}.")
  1211. # try:
  1212. # response_data = response.json()
  1213. # # Extract the assistant's reply
  1214. # assistant_reply = response_data['choices'][0]['message']['content'].strip()
  1215. # logging.info(f"Assistant's reply from {model_name}: {assistant_reply}")
  1216. # return assistant_reply
  1217. # except (KeyError, IndexError) as e:
  1218. # logging.error(f"Error extracting assistant's reply from {model_name}: {e}")
  1219. # logging.debug(f"Full response from {model_name}: {json.dumps(response_data, indent=4)}")
  1220. # return "Error extracting reply."
  1221. # elif response.status_code in [429, 500, 502, 503, 504]:
  1222. # # Retry for rate limiting or server errors
  1223. # logging.warning(f"Attempt {attempt} failed with status code {response.status_code}. Retrying in {backoff} seconds...")
  1224. # time.sleep(backoff)
  1225. # backoff *= 2
  1226. # else:
  1227. # logging.error(f"Request to {model_name} failed with status code {response.status_code}")
  1228. # logging.error(f"Response: {response.text}")
  1229. # return f"Error: Received status code {response.status_code}"
  1230. # except requests.exceptions.RequestException as e:
  1231. # logging.error(f"Attempt {attempt} failed with exception: {e}")
  1232. # if attempt < retries:
  1233. # logging.info(f"Retrying in {backoff} seconds...")
  1234. # time.sleep(backoff)
  1235. # backoff *= 2
  1236. # else:
  1237. # logging.error("All retry attempts failed.")
  1238. # return f"Error: {e}"
  1240. # return "Error: Unable to retrieve response."
  1242. # def main():
  1243. # # Define the prompt (no image URL since all models are text-only)
  1244. # prompt = "What is the average age Parkinson's disease starts?"
  1246. # # Prepare a list to hold the responses
  1247. # responses = []
  1249. # for model in MODELS.keys():
  1250. # reply = send_request(model, prompt)
  1251. # responses.append({
  1252. # "Model": model,
  1253. # "Response": reply
  1254. # })
  1256. # # Create a pandas DataFrame to display the responses
  1257. # df = pd.DataFrame(responses)
  1259. # # Debug: Verify the DataFrame structure
  1260. # logging.debug("DataFrame before pivoting:")
  1261. # logging.debug(df)
  1263. # # Check if 'Model' and 'Response' columns exist
  1264. # if 'Model' not in df.columns or 'Response' not in df.columns:
  1265. # logging.error("DataFrame missing 'Model' or 'Response' columns.")
  1266. # exit(1)
  1268. # # Check for any None or NaN values in 'Model' column
  1269. # if df['Model'].isnull().any():
  1270. # logging.error("Found None or NaN values in 'Model' column.")
  1271. # exit(1)
  1273. # # Set 'Model' as the index and transpose the DataFrame
  1274. # df_pivot = df.set_index('Model').transpose()
  1276. # # Display the DataFrame
  1277. # print("\n=== LLM Responses ===")
  1278. # print(df_pivot.to_string(index=False))
  1280. # # Optional: Using PrettyTable for better console display
  1281. # try:
  1282. # from prettytable import PrettyTable
  1283. # table = PrettyTable()
  1284. # table.field_names = ["Model", "Response"]
  1286. # for resp in responses:
  1287. # table.add_row([resp["Model"], resp["Response"]])
  1289. # print("\n=== LLM Responses (PrettyTable) ===")
  1290. # print(table)
  1291. # except ImportError:
  1292. # logging.warning("PrettyTable not installed. Skipping table formatting.")
  1294. # # Optional: Using colorama for color-coded console output
  1295. # if COLORAMA_AVAILABLE:
  1296. # print("\n=== LLM Responses (Color-Coded) ===")
  1297. # for resp in responses:
  1298. # color = MODEL_COLORS.get(resp["Model"], Fore.WHITE)
  1299. # print(f"{color}{resp['Model']}: {resp['Response']}{Style.RESET_ALL}")
  1300. # else:
  1301. # logging.info("Colorama not available. Skipping color-coded output.")
  1303. # if __name__ == "__main__":
  1304. # main()
  1306. # %%
  1309. # %%
  1312. # %%
  1313. import requests
  1317. resp = requests.post('https://textbelt.com/text', {
  1319. 'phone': '9163802941',
  1321. 'message': 'LLM Processing Done',
  1323. 'key': '138adc496234ca311154757db147f552afa8ba83FfrCKJ36kTJNXq65nlsvvF4Pu',
  1325. })
  1327. print(resp.json())
  1329. # %%
  1332. # %%
  1335. # %%
  1338. # %%
  1341. # %%
  1344. # %%
  1347. # %%
  1350. # %%
  1351. import os
  1352. import requests
  1353. import json
  1354. import logging
  1356. # Configure logging
  1357. logging.basicConfig(level=logging.INFO)
  1359. # Configuration Variables
  1360. API_KEY = openrouter.api_key # Ensure this environment variable is set
  1361. SITE_URL = 'https://pd-research.com' # Your site's URL
  1362. APP_NAME = 'PD Research' # Your application's name
  1364. def send_test_prompt():
  1365. logging.info("Starting the API request...")
  1366. try:
  1367. # Prepare the payload
  1368. payload = {
  1369. "model": "openai/o1-mini",
  1371. "messages": [
  1372. {"role": "user", "content": "What is the average age Parkinson's disease starts?"}
  1373. ],
  1374. "top_p": 1,
  1375. "temperature": 1,
  1376. "frequency_penalty": 0,
  1377. "presence_penalty": 0,
  1378. "repetition_penalty": 1,
  1379. "top_k": 0,
  1380. }
  1382. # Make the POST request to the OpenRouter API
  1383. response = requests.post(
  1384. url="https://openrouter.ai/api/v1/chat/completions",
  1385. headers={
  1386. "Authorization": f"Bearer {API_KEY}",
  1387. "HTTP-Referer": SITE_URL, # Optional: Include if you want your app featured
  1388. "X-Title": APP_NAME, # Optional: Shows your app name on OpenRouter rankings
  1389. },
  1390. data=json.dumps(payload)
  1391. )
  1393. logging.info("API request sent. Awaiting response...")
  1395. # Check if the request was successful
  1396. if response.status_code == 200:
  1397. logging.info("Received successful response from the API.")
  1398. response_data = response.json()
  1400. # Print the full JSON response for debugging
  1401. logging.debug("Full Response JSON:")
  1402. logging.debug(json.dumps(response_data, indent=4))
  1404. # Extract the assistant's reply
  1405. try:
  1406. assistant_reply = response_data['choices'][0]['message']['content'].strip()
  1407. logging.info(f"Assistant's reply: {assistant_reply}")
  1408. except (KeyError, IndexError) as e:
  1409. logging.error("Error extracting assistant's reply:", exc_info=True)
  1410. logging.debug("Full response:", json.dumps(response_data, indent=4))
  1411. else:
  1412. logging.error(f"Request failed with status code {response.status_code}")
  1413. logging.error(f"Response: {response.text}")
  1415. except requests.exceptions.RequestException as e:
  1416. logging.error("An error occurred while making the request:", exc_info=True)
  1418. # --- Execute the Test Prompt ---
  1419. if __name__ == "__main__":
  1420. send_test_prompt()
  1422. # %%
  1425. # %%
  1428. # %%
  1431. # %%
  1434. # %%
  1435. ß
  1437. # %%
  1440. # %%
  1443. # %%
  1446. # %%
  1449. # %%
  1452. # %%
  1453. 1=2
  1456. # %%
  1459. # %%
  1462. # %%
  1465. # %%
  1468. # %%
  1471. # %%
  1472. import pandas as pd
  1473. import openai
  1475. # Set your OpenAI API key
  1476. # openai.api_key = 'YOUR_OPENAI_API_KEY' # Replace with your actual API key
  1478. # Function to query OpenAI for each row
  1479. def get_brain_area(row):
  1480. # Prepare the prompt with the relevant data from the row
  1481. prompt = f"""Given the following study data:
  1483. {row.to_json()}
  1485. What area of the brain are they looking for?"""
  1487. try:
  1488. response = openai.chat.completions.create(
  1489. model='gpt-3.5-turbo', # Use 'gpt-4' if you have access
  1490. messages=[
  1491. {"role": "user", "content": prompt}
  1492. ],
  1493. max_tokens=100,
  1494. temperature=0.5,
  1495. )
  1496. # Extract the assistant's reply
  1497. answer = response.choices[0].message.content.strip()
  1498. # Print the response
  1499. print(f"Row {row.name} response: {answer}")
  1500. return answer
  1501. except Exception as e:
  1502. print(f"Error processing row {row.name}: {e}")
  1503. return None
  1505. # Ensure that 'filtered_df_covariates' is defined with your DataFrame
  1506. # For example:
  1507. # filtered_df_covariates = pd.read_csv('your_data.csv')
  1509. # Apply the function to each row and save the result in a new column
  1510. filtered_df_covariates['BrainArea'] = filtered_df_covariates.apply(get_brain_area, axis=1)
  1512. # %%
  1513. import pandas as pd
  1514. import openai
  1516. # Set your OpenAI API key
  1517. # openai.api_key = 'YOUR_OPENAI_API_KEY' # Replace with your actual API key
  1519. # Function to query OpenAI for each row
  1520. def get_brain_area(row):
  1521. # Prepare the prompt with the relevant data from the row
  1522. prompt = f"""Given the following study data:
  1524. {row.to_json()}
  1526. What area of the brain are they looking for?"""
  1528. try:
  1529. response = openai.chat.completions.create(
  1530. model='gpt-3.5-turbo', # Use 'gpt-4' if you have access
  1531. messages=[
  1532. {"role": "user", "content": prompt}
  1533. ],
  1534. max_tokens=100,
  1535. temperature=0.5,
  1536. )
  1537. # Extract the assistant's reply
  1538. answer = response.choices[0].message.content.strip()
  1539. # Print the response
  1540. print(f"Row {row.name} response: {answer}")
  1541. return answer
  1542. except Exception as e:
  1543. print(f"Error processing row {row.name}: {e}")
  1544. return None
  1546. # Ensure that 'filtered_df_covariates' is defined with your DataFrame
  1547. # For example:
  1548. # filtered_df_covariates = pd.read_csv('your_data.csv')
  1550. # Apply the function to each row and save the result in a new column
  1551. filtered_df_covariates['BrainArea'] = filtered_df_covariates.apply(get_brain_area, axis=1)
  1553. # %%
  1556. # %%
  1557. # import pandas as pd
  1558. # import openai
  1560. # # Set your OpenAI API key
  1562. # # Function to query OpenAI for each row
  1563. # def get_brain_area(row):
  1564. # # Prepare the prompt with the relevant data from the row
  1565. # prompt = f"""Given the following study data:
  1567. # {row.to_json()}
  1569. # What area of the brain are they looking for?"""
  1571. # try:
  1572. # response = openai.chat.completions.create(
  1573. # model='gpt-3.5-turbo', # Use 'gpt-4' if you have access
  1574. # messages=[
  1575. # {"role": "user", "content": prompt}
  1576. # ],
  1577. # max_tokens=100,
  1578. # temperature=0.5,
  1579. # )
  1580. # # Extract the assistant's reply
  1581. # answer = response.choices[0].message.content.strip()
  1582. # return answer
  1583. # except Exception as e:
  1584. # print(f"Error processing row {row.name}: {e}")
  1585. # return None
  1587. # # Ensure that 'filtered_df_covariates' is defined with your DataFrame
  1588. # # For example:
  1589. # # filtered_df_covariates = pd.read_csv('your_data.csv')
  1591. # # Apply the function to each row and save the result in a new column
  1592. # filtered_df_covariates['BrainArea'] = filtered_df_covariates.apply(get_brain_area, axis=1)
  1594. # %%
  1597. # %%
  1600. # %%
  1603. # %%
  1606. # %%
  1609. # %%
  1612. # %%
  1615. # %%
  1618. # %%
  1621. # %%
  1622. import openai
  1623. print(openai.__version__)
  1626. # %%
  1629. # %%
  1632. # %%
  1635. # %%
  1638. # %%
  1639. def _call_local_llama(self, prompt: str) -> str:
  1640. """Call local Llama instance using Ollama API"""
  1641. try:
  1642. # Construct the API endpoint
  1643. api_endpoint = f"{self.local_url}generate"
  1646. # Prepare the request payload
  1647. payload = {
  1648. "model": self.local_model,
  1649. "prompt": prompt,
  1650. "options": {
  1651. "temperature": 0.1
  1652. }
  1653. }
  1655. # Send the POST request
  1656. response = requests.post(api_endpoint, json=payload)
  1658. if response.status_code == 200:
  1659. # Extract the generated text from the response
  1660. result = response.json()
  1661. if 'generated_text' in result:
  1662. return result['generated_text']
  1663. else:
  1664. print(f"Unexpected response structure: {result}")
  1665. return ''
  1666. else:
  1667. raise Exception(f"Local LLM error: Status {response.status_code} - {response.text}")
  1669. except requests.exceptions.RequestException as e:
  1670. raise Exception(f"Network error calling Ollama: {str(e)}")
  1671. except json.JSONDecodeError as e:
  1672. raise Exception(f"Error parsing Ollama response: {str(e)}")
  1673. except Exception as e:
  1674. raise Exception(f"Error calling local Llama: {str(e)}")
  1676. # %%
  1679. # %%
  1680. import openai
  1682. # Your data
  1683. data_json = '{"study": "example study data"}'
  1684. prompt = f"Given the following study data, provide detailed information about the type of study and the brain area involved:\n\n{data_json}"
  1686. # Make the API request to OpenAI
  1687. response = openai.ChatCompletion.create(
  1688. model="gpt-4",
  1689. messages=[
  1690. {"role": "system", "content": "You are a helpful assistant."},
  1691. {"role": "user", "content": prompt}
  1692. ],
  1693. max_tokens=1500,
  1694. n=1,
  1695. stop=None,
  1696. temperature=0.5,
  1697. )
  1699. # Print the response
  1700. print(response['choices'][0]['message']['content'].strip())

03_LLM_processor.ipynb at commit 2e829ad, under MIT · at the source

Overview

Authors: Richard J Young1, Jorge Fonseca2, Brach Poston1,3
ORCID iDs: Richard J Young, Jorge Fonseca
  1. Interdisciplinary Neuroscience, University of Nevada, Las Vegas, NV 89154, USA
  2. Department of Computer Science, Howard R. Hughes College of Engineering, University of Nevada, Las Vegas, NV 89154, USA
  3. Department of Kinesiology and Nutrition Sciences, University of Nevada, Las Vegas, NV 89154, USA
Institutions: University of Nevada, Las Vegas (United States)
Journal: Bioengineering (Basel, Switzerland), volume 13, issue 7, article 773
Dates: received 27 May 2026; accepted 28 June 2026; published online 2 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.3390/bioengineering13070773 · PMID 42510439 · PMCID PMC13404533 · OpenAlex W7167097198
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: other (modality), human (organism), Parkinson's (population), clinical / translational (subfield)
Methods: Connectivity, Statistics, Machine learning
Keywords: clinical trial data extraction, large language models, neuromodulation, transcranial direct current stimulation, API integration, systematic review methodology, Parkinson’s disease, evidence synthesis, research automation
Topic: Machine Learning in Healthcare (Artificial Intelligence, Computer Science), according to OpenAlex
Citations: not cited yet (Europe PMC); 115 references in the paper

Abstract

(1) Background: Clinical trial data extraction from registries such as ClinicalTrials.gov remains labor-intensive and error-prone, often missing critical details hidden in unstructured protocol descriptions. Large Language Models (LLMs) offer potential to automate this process, yet systematic multi-model comparisons on real clinical trial data remain scarce. (2) Methods: Four LLMs (OpenAI o4-mini-high, Anthropic Claude-Sonnet-4, Google Gemini 2.5-Pro, and Meta Llama-4-Maverick) extracted brain stimulation parameters from 67 transcranial direct current stimulation (tDCS) trials in Parkinson’s disease via a structured JSON schema. Pairwise inter-model agreement was quantified with Cohen’s Kappa and percentage agreement across binary, categorical, and multi-component task tiers. (3) Results: Under exact-string matching, agreement was near-perfect for binary classifications (non-invasive classification: 100%; brain stimulation presence: 99.3%, κ = 0.50) and substantial for categorical extractions (primary stimulation type: 96.4%, κ = 0.70), but fell to 48.6% (κ = 0.43) for complex anatomical targets. Numeric parameters revealed model-specific strengths: o4-mini-high and Claude-Sonnet-4 achieved perfect duration agreement (r = 1.000, n = 19) while Llama-4-Maverick diverged substantially (r < 0.12). Validation against an expert gold standard (100% inter-annotator agreement on a 20-trial overlap) confirmed high extraction accuracy across all features (mean 93.7–98.9%). Crucially, the low agreement on anatomical targets proved to be an artifact of exact-string scoring: under the same semantic matching used to measure accuracy, inter-model agreement rose to 97.0%, coinciding with the 95.5% expert accuracy. Inter-model agreement therefore tracks accuracy once both are measured on a common basis. (4) Conclusions: Exact-string inter-model agreement decreases with task complexity, but this decline largely reflects interchangeable free-text wording rather than reduced accuracy. Evaluated semantically, agreement and expert accuracy are both high and closely aligned. A residual risk is not low accuracy but the rare error shared across all models, which agreement cannot detect, and which overall accuracy can itself mask when one class dominates. These findings inform hybrid human–AI systematic review pipelines in which targeted expert oversight focuses on shared-error and minority-class detection.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 14 matches between paragraphs and lines of code.

ricyoung/ClinicalTrialLLM-Extractor

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 2e829ad114ef2a1ea3f8e48732d8ab6fe5fc5138, 1 July 2026
Languages: Jupyter (9), Python (1)
Size: 36 files, 10 scripts
Software Heritage: not archived
Found in: “Data Availability Statement”
Holds: README, license file, environment (requirements.txt), 9 notebooks
Not found: CITATION.cff, tests, continuous integration, documentation
Tools: NumPy (9 files), pandas (9 files), Matplotlib (6 files), scikit-learn (6 files), Pingouin (2 files)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
12 files
At the source: github.com/ricyoung/ClinicalTrialLLM-Extractor

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 10 scripts, each with its path and the digest of its content;
  • 14 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

All clinical trial data are publicly available from ClinicalTrials.gov. Processed datasets, analysis code, and the complete extraction pipeline are available at: https://github.com/ricyoung/ClinicalTrialLLM-Extractor (accessed on 27 June 2026) (Appendix C). The normalization lexicon, gold-standard annotations, and annotation codebook are also provided as Supplementary Materials (Files S1–S4).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 3 authors, 9 keywords, 112 references.

Cite

This paper

Young, R. J., Fonseca, J., & Poston, B. (2026). From API to Action: A Multi-Model Comparison of OpenAI, Anthropic, Google, and Meta LLMs for Clinical Trial Data Extraction. Bioengineering (Basel, Switzerland), 13(7), 773. https://doi.org/10.3390/bioengineering13070773

BibTeX

@article{young2026api,
author = {Young, Richard J and Fonseca, Jorge and Poston, Brach},
title = {{From API to Action: A Multi-Model Comparison of OpenAI, Anthropic, Google, and Meta LLMs for Clinical Trial Data Extraction}},
journal = {Bioengineering (Basel, Switzerland)},
year = {2026},
month = jul,
volume = {13},
number = {7},
pages = {773},
publisher = {Multidisciplinary Digital Publishing Institute (MDPI)},
issn = {2306-5354},
doi = {10.3390/bioengineering13070773},
url = {https://doi.org/10.3390/bioengineering13070773},
pmid = {42510439},
pmcid = {PMC13404533}
}

RIS

TY - JOUR
AU - Young, Richard J
AU - Fonseca, Jorge
AU - Poston, Brach
TI - From API to Action: A Multi-Model Comparison of OpenAI, Anthropic, Google, and Meta LLMs for Clinical Trial Data Extraction
T2 - Bioengineering (Basel, Switzerland)
J2 - Bioengineering (Basel)
PY - 2026
DA - 2026/07/02
VL - 13
IS - 7
SP - 773
SN - 2306-5354
PB - Multidisciplinary Digital Publishing Institute (MDPI)
DO - 10.3390/bioengineering13070773
UR - https://doi.org/10.3390/bioengineering13070773
LA - en
ER -

CSL-JSON

{
"id": "10.3390/bioengineering13070773",
"type": "article-journal",
"title": "From API to Action: A Multi-Model Comparison of OpenAI, Anthropic, Google, and Meta LLMs for Clinical Trial Data Extraction",
"container-title": "Bioengineering (Basel, Switzerland)",
"author": [
{
"family": "Young",
"given": "Richard J"
},
{
"family": "Fonseca",
"given": "Jorge"
},
{
"family": "Poston",
"given": "Brach"
}
],
"container-title-short": "Bioengineering (Basel)",
"volume": "13",
"issue": "7",
"page": "773",
"DOI": "10.3390/bioengineering13070773",
"PMID": "42510439",
"PMCID": "PMC13404533",
"ISSN": "2306-5354",
"publisher": "Multidisciplinary Digital Publishing Institute (MDPI)",
"URL": "https://doi.org/10.3390/bioengineering13070773",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
2
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1162/nol.a.244 [code]
A Novel Approach to Map the Causal Impact of Brain Stimulation on Semantic Processing With Language Models.
Journal: Neurobiology of language (Cambridge, Mass.)
In common: Pingouin, scikit-learn, Matplotlib, 1 other tool, other, 1 reference
[2] doi:10.1038/s41531-026-01380-1 [code]
Identifying maximal beta power from directional subthalamic local field potentials in Parkinson's disease.
Journal: NPJ Parkinson's disease
In common: Pingouin, scikit-learn, pandas, 2 other tools, Parkinson's, clinical / translational
[3] doi:10.1002/alz.71649 [code]
Postmortem brain MRI reveals differential associations of subcortical and limbic volumes with cortical thinning and neurodegenerative pathologies.
Journal: Alzheimer's & dementia : the journal of the Alzheimer's Association
In common: Pingouin, scikit-learn, pandas, 2 other tools, Parkinson's
[4] doi:10.1093/braincomms/fcag218 [code]
Magnetoencephalography biomarkers for assessing myelin content and neuronal function in acute optic neuritis.
Journal: Brain communications
In common: Pingouin, scikit-learn, pandas, 2 other tools, other
[5] doi:10.1016/j.nicl.2026.104012 [code]
Structural-functional multilayer brain network properties and outcome of combined repetitive transcranial magnetic stimulation and psychotherapy for obsessive-compulsive disorder.
Journal: NeuroImage. Clinical
In common: Pingouin, scikit-learn, pandas, 2 other tools, other
[6] doi:10.7554/elife.100605 [code]
Age-related changes in ‘cortical’ 1/f dynamics are linked to cardiac activity
Journal: n/a
In common: Pingouin, scikit-learn, pandas, 2 other tools, other
[7] doi:10.1038/s41398-026-04081-8 [code]
Functional system-specific brain aging across the Alzheimer's disease continuum.
Journal: Translational psychiatry
In common: Pingouin, scikit-learn, pandas, 2 other tools, clinical / translational
[8] doi:10.1016/j.patter.2026.101538 [code]
A multi-modal foundation model for brain disease diagnosis and medical imaging.
Journal: Patterns (New York, N.Y.)
In common: scikit-learn, pandas, Matplotlib, 1 other tool, clinical / translational, 1 reference
[9] doi:10.1162/imag.a.1259 [code]
Neuronal avalanches as a predictive biomarker for guiding tailored BCI training programs.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: Pingouin, scikit-learn, pandas, 2 other tools, clinical / translational
[10] doi:10.1371/journal.pbio.3003856 [code]
Aging and metabolism contribute separately to brain-body health.
Journal: PLoS biology
In common: scikit-learn, pandas, Matplotlib, 1 other tool, clinical / translational, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.