Merge branch 'master' of wiscigt.powertheword.com:oil/iraemm

README.md

@@ -2,6 +2,74 @@
 
 Manage images and data related to irAE project.
 
+# Slicer module
+A Slicer module was created to assist Radiology and Nuclear Medicine phsicians in
+reviewing the images. The following lists the installation, setup and usage of the module.
+
+## Installation
+Download the [code][iraemm] and [dependencies][SlicerLabkeyExtension]. Unzip. To
+have Slicer know where the files are, open Slicer, and under Edit->Application settings 
+select Modules section. Under Paths, click on Add and navigate to newly unzipped
+directories. We need `labkeyBrowser` and `DICOMtools` from `SlcierLabkeyExtension` and 
+`slicerModule` from `iraeMM` code. After clicking `OK`, Slicer will and has to
+be restarted.
+
+## Setup
+To access LabKey, the Slicer tools must be configured. Do that by selecting LabKey->labkeyBrowser
+module from module list and fill appropriate fields.
+
+### Onko-nix
+For accessing OIL internal site, the settings are:
+- Server: `http://onko-nix.onko-i.si:8080
+- Labkey username: username, given at the LabKey site, typically your email
+- Labkey password: password, generated when accessing LabKey site
+
+The rest needn't be changed. 
+
+### Setup verification and storage
+Once the data is entered, click on Init to check whether LabKey can be accessed. If
+the button turns green, you are OK. Do `Save configuration`. 
+
+## Usage
+Use Labkey->iraemmBrowser module. The `Patients` section lets you select the patient
+and corresponding visit. On `Load` the data gets loaded from the server. 
+
+### Converting segmentations
+The segmentations on server are stored as label maps, which have to be 
+converted to segments for Slicer. To do that, select `Segmentations` module 
+from the drop-down menu. Under Active segmentations, a new segmentation must
+be created by selecting `Create new segmentation` option from the pull down menu. Scroll down 
+to `Export/import models and labelmaps` and change the mode to `Import` by 
+moving the radio button selection. The Input type should be set to `labelmap`. Input
+node should match selected patient/visit pair and should end in `Segm`. Click on `Import` 
+button further down. 
+
+### Removing labelmap volume
+The source labelmap volume will obscure other volumes, so we should delete it. Do that
+by selecting `Volumes` module and rotating˛`Active volume` to point to labelmap
+used in segmentation creation. Once selected, select `Delete current volume` from
+the same pull down menu next to Active volume.
+
+### Viewing segmentations
+Labelmap was converted to a set of Segments, which are listed in the `Segmentations` 
+module. By clicking on the open/closed eye icon, a segment can be made visible or invisible
+on the windows. Further setup can be made in the `View controls` module, where
+visibility and mode of each volume can be adjusted. For the segmentations, which
+appear in the top layer, either continuous, continuous with sharpened edges or edge 
+only mode are available by clicking on display icon.
+
+## Entering review
+The module has a review section. Select LabKey->iraemmBrowser and navigate to review
+secion. Four levels of agreement can be selected and an addition Comments field is 
+available. Once filled, review is submitted by clicking on Submit button. Choices can be
+changed any time and new values can be pushed to database with further clicks of
+the button. 
+
+## Clearing
+Once done, a patient should be cleared to minimize interference in segmentation 
+evaluation. Do that by pressing `Clear` in `Patients` section of the `iraeMMBrowser` module. 
+
+
 ### Dependencies 
 To access LabKey, the [python API][labkeyInterface] was used.
 Anonymization and NIfTI conversion are based on phenomenal [nibabel][] tools. 
@@ -12,7 +80,8 @@ Data storage is provided by [Orthanc][] with associated [interface][orthancInter
 Anonymization must be run as a `tomcat8` user for access to data files. Check setup 
 in the `anonymization.py` and run it with `runPython.sh anonymization.py`. 
 
-
+[SlicerLabkeyExtension]:http://wiscigt.powertheword.com/labkey/SlicerLabkeyExtension/-/archive/SlicerExtensionIndex/SlicerLabkeyExtension-SlicerExtensionIndex.zip
+[iraemm]: http://wiscigt.powertheword.com/oil/iraemm/-/archive/master/iraemm-master.zip
 [nibabel]: https://nipy.org/nibabel/gettingstarted.html
 [labkeyInterface]: http://wiscigt.powertheword.com/andrej.studen/labkeyInterface
 [Orthanc]:https://www.orthanc-server.com

pythonScripts/addSegmentations.py

@@ -0,0 +1,110 @@
+import os
+import json
+import re
+import subprocess
+import nibabel
+import shutil
+import sys
+
+shome=os.path.expanduser('~nixUser')
+fhome=os.path.expanduser('~')
+with open(os.path.join(fhome,".labkey","setup.json")) as f:
+    setup=json.load(f)
+
+sys.path.insert(0,setup["paths"]["labkeyInterface"])
+import labkeyInterface
+import labkeyDatabaseBrowser
+import labkeyFileBrowser
+
+#sys.path.insert(1,shome+'/software/src/IPNUMM/dicomUtils')
+#import loadDicom
+
+fconfig=os.path.join(fhome,'.labkey','network.json')
+
+net=labkeyInterface.labkeyInterface()
+net.init(fconfig)
+db=labkeyDatabaseBrowser.labkeyDB(net)
+fb=labkeyFileBrowser.labkeyFileBrowser(net)
+
+project='iPNUMMretro/Study'
+dataset='Imaging1'
+tempBase=os.path.join(fhome,'temp')
+
+#all images from database
+ds=db.selectRows(project,'study',dataset,[])
+#imageSelector={"CT":"CT","PET":"PETWB"};
+imageResampledField={"Segm":"Segmentation"}
+
+#projectNIfTIBase=os.path.join(labkeyBase,'files',project,'@files/nifti')
+#use webdav to transfer file (even though it is localhost)
+
+
+def getPatientLabel(row):
+    return row['PatientId'].replace('/','_') 
+
+def getVisitLabel(row):
+    return 'VISIT_'+str(int(row['SequenceNum']))
+
+def getStudyLabel(row):
+    return getPatientLabel(row)+'-'+getVisitLabel(row)
+
+def updateRow(project,dataset,row,imageResampledField,gzFileNames):
+    for im in imageResampledField:
+        row[imageResampledField[im]]=gzFileNames[im]
+    db.modifyRows('update',project,'study',dataset,[row])
+ 
+i=0
+for row in ds["rows"]:
+
+    #interesting files are processedDir/studyName_CT_notCropped_2mmVoxel.nii
+    #asn processedDir/studyName_PET_notCropped_2mmVoxel.nii
+    gzFileNames={im:\
+            getStudyLabel(row)+'_'+im+'.nii.gz'\
+                for im in imageResampledField}
+    
+    #build/check remote directory structure
+    remoteDir=fb.buildPathURL(project,\
+            ['preprocessedImages',getPatientLabel(row),getVisitLabel(row)])
+
+    gzRemoteFiles={im:remoteDir+'/'+f\
+            for (im,f) in gzFileNames.items()}
+
+    remoteFilePresent=[fb.entryExists(f)\
+            for f in gzRemoteFiles.values()]
+
+    for f in gzRemoteFiles.values():
+        print("[{}]: [{}]".format(f,fb.entryExists(f)))
+
+
+    if all(remoteFilePresent):
+        print("Entry for row done.")
+        updateRow(project,dataset,row,imageResampledField,\
+                gzFileNames)
+        continue
+
+    inputDir=fb.buildPathURL(project,['segmentations']) 
+    inputFiles={im:inputDir+'/'+f for (im,f) in gzFileNames.items()}
+    
+    for im in inputFiles:
+        f=inputFiles[im]
+        if not fb.entryExists(f):
+            print("Input file {} not found".format(f))
+            continue
+        print("Found {}".format(f))
+        localFile=os.path.join(tempBase,gzFileNames[im])
+        print("Local {}".format(localFile))
+        fb.readFileToFile(f,localFile)
+        fb.writeFileToFile(localFile,gzRemoteFiles[im])
+        print("Remote {}".format(gzRemoteFiles[im]))
+        os.remove(localFile)
+
+    #update row and let it know where the processed files are
+    updateRow(project,dataset,row,imageResampledField,gzFileNames)
+   
+
+    if i==-1:
+        break
+    i=i+1
+
+
+print("Done")

pythonScripts/preprocess.py

@@ -8,15 +8,15 @@ import sys
 
 shome=os.path.expanduser('~nixUser')
 fhome=os.path.expanduser('~')
-with open(os.path.join(fhome,".labkey","setup.json"),"w") as f:
+with open(os.path.join(fhome,".labkey","setup.json")) as f:
     setup=json.load(f)
 
-sys.path.insert(1,setup["paths"]["labkeyInterface"])
+sys.path.insert(0,setup["paths"]["labkeyInterface"])
 import labkeyInterface
 import labkeyDatabaseBrowser
 import labkeyFileBrowser
 
-sys.path.insert(1,setup["paths"]["orthancInterface"])
+sys.path.insert(0,setup["paths"]["orthancInterface"])
 import orthancInterface
 import orthancFileBrowser
 
@@ -47,13 +47,13 @@ ofb=orthancFileBrowser.orthancFileBrowser(onet)
 hi=0
 project='iPNUMMretro/Study'
 dataset='Imaging'
-h
 tempBase=os.path.join(fhome,'temp')
 
 #all images from database
 ds=db.selectRows(project,'study',dataset,[])
-imageSelector={"CT":"CT","PETWB":"PET"};
-imageResampledField={"CT":"ctResampled","PETWB":"petResampled"}
+#imageSelector={"CT":"CT","PET":"PETWB"};
+imageSelector={"CT":"CT_orthancId","PET":"PETWB_orthancId"}
+imageResampledField={"CT":"ctResampled","PET":"petResampled"}
 
 #projectNIfTIBase=os.path.join(labkeyBase,'files',project,'@files/nifti')
 #use webdav to transfer file (even though it is localhost)
@@ -92,12 +92,12 @@ def getDicom(ofb,row,zipDir,rawDir,im,imageSelector):
 
     #Return True for valid outcome and False for troubles in row formating or unzip failures
 
-    seriesId=row[im];
+    seriesId=row[imageSelector[im]];
     if seriesId=="0":
         return False
 
     print("{}: {}".format(im,seriesId))
-    fname=os.path.join(zipDir,getStudyLabel(row)+'_'+imageSelector[im]+".zip");
+    fname=os.path.join(zipDir,getStudyLabel(row)+'_'+im+".zip");
 
     #copy data from orthanc
     if os.path.isfile(fname):
@@ -108,7 +108,7 @@ def getDicom(ofb,row,zipDir,rawDir,im,imageSelector):
 
     #unzip the zipped dicom series
 
-    unzipDir=os.path.join(rawDir,imageSelector[im])
+    unzipDir=os.path.join(rawDir,im)
 
     if os.path.isdir(unzipDir):
         print("Data already unzipped")
@@ -142,7 +142,7 @@ for row in ds["rows"]:
     #interesting files are processedDir/studyName_CT_notCropped_2mmVoxel.nii
     #asn processedDir/studyName_PET_notCropped_2mmVoxel.nii
     volumeFileNames={im:\
-            getStudyLabel(row)+'_'+imageSelector[im]+
+            getStudyLabel(row)+'_'+im+
             '_notCropped_2mmVoxel.nii'\
                 for im in imageSelector}
     gzFileNames={im:f+".gz" \
@@ -187,7 +187,7 @@ for row in ds["rows"]:
 
     #specify local file names with path 
     volumeFiles={im:os.path.join(processedDir,f)\
-            for (im.f) in volumeFileNames.items()}
+            for (im,f) in volumeFileNames.items()}
     gzFiles={im:f+".gz"\
             for (im,f) in volumeFiles.items()}