I'm trying to implement a custom script execution system using GroovyScriptEngine that loads scripts from an S3-like storage system. However, I'm encountering issues with dependency resolution. Here's my setup:
I have a custom S3FileSystemService that mimics S3 storage. I've implemented a custom ResourceConnector for GroovyScriptEngine. I'm using a custom GroovyClassLoader. My file structure in S3 is:
The content of UseHelper.groovy is:
def helper = new Helper()
helper.getMessage().toUpperCase()
The content of Helper.groovy is:
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
When I try to execute UseHelper.groovy, I get the following error:
startup failed:
file:Helper.groovy: Helper.groovy (No such file or directory)
Here's my ResourceConnector implementation
class S3ResourceConnector implements ResourceConnector {
private final S3FileSystemService s3FileSystemService
private final String clientKey
S3ResourceConnector(S3FileSystemService s3FileSystemService, String clientKey) {
this.s3FileSystemService = s3FileSystemService
this.clientKey = clientKey
}
@Override
URLConnection getResourceConnection(String resourceName) throws ResourceException {
try {
println "Original resourceName: ${resourceName}"
// Remove "file:" prefix if present
if (resourceName.startsWith("file:")) {
resourceName = resourceName.substring(5)
println "ResourceName after removing 'file:' prefix: ${resourceName}"
}
// Remove leading "root/scripts/" if present
if (resourceName.startsWith("root/scripts/")) {
resourceName = resourceName.substring("root/scripts/".length())
println "ResourceName after removing 'root/scripts/': ${resourceName}"
}
// Handle dependency after '$' if present
if (resourceName.contains('$')) {
resourceName = resourceName.substring(resourceName.indexOf('$') + 1)
println "ResourceName after handling dependency: ${resourceName}"
}
// If the resource name doesn't end with .groovy, append it
if (!resourceName.endsWith(".groovy")) {
resourceName += ".groovy"
println "ResourceName after appending .groovy: ${resourceName}"
}
String fullPath = "root/scripts/" + resourceName
println "Attempting to load resource: ${fullPath}"
String content = s3FileSystemService.getFileContent(clientKey, fullPath)
println "Content loaded successfully for ${fullPath}"
println "Content: ${content}"
return new ByteArrayURLConnection(resourceName, content.getBytes())
} catch (Exception e) {
println "Failed to load resource: ${resourceName}"
e.printStackTrace()
throw new ResourceException("Failed to load resource: " + resourceName, e)
}
}
}
And here's my test method:
@Test
void testScriptExecutionWithS3FolderStructure() throws Exception {
// Get the folder structure from S3
FileNode rootNode = s3FileSystemService.getRootFileSystem(CLIENT_KEY)
println "Root node structure:"
printFileStructure(rootNode, 0)
// List all objects in the bucket for debugging
listAllObjects()
// Create a custom GroovyClassLoader
S3GroovyClassLoader classLoader = new S3GroovyClassLoader(this.class.classLoader, s3FileSystemService, CLIENT_KEY)
// Create a custom GroovyScriptEngine that uses our S3 folder structure
GroovyScriptEngine engine = new GroovyScriptEngine(new S3ResourceConnector(s3FileSystemService, CLIENT_KEY), classLoader)
println "Attempting to run UseHelper.groovy"
// Execute the script
Binding binding = new Binding()
Object result = engine.run("UseHelper.groovy", binding)
println "Script execution result: ${result}"
// Assert the result
assertEquals("HELLO FROM HELPER CLASS", result)
}
The scripts are successfully loaded from S3, but GroovyScriptEngine fails to compile Helper.groovy when it's referenced in UseHelper.groovy.
How can I modify my setup to correctly resolve and compile dependent scripts in this S3-based system?
Note: I can get all of this to work by simply adding engine.loadScriptByName("Helper.groovy") before initialising the GSE, but I would like the gse to dynamically be able to handle this.
In case this helps, here are the logs from the test execution before the failure
File system structure updated in S3
Root node structure:
root (folder)
Listing all objects in the bucket:
- testClient/root/
- testClient/root/scripts/
- testClient/root/scripts/Helper.groovy
- testClient/root/scripts/UseHelper.groovy
Attempting to run UseHelper.groovy
Original resourceName: UseHelper.groovy
Attempting to load resource: root/scripts/UseHelper.groovy
Attempting to fetch S3 object with key: testClient/root/scripts/UseHelper.groovy
Content loaded successfully for root/scripts/UseHelper.groovy
Content:
def helper = new Helper()
helper.getMessage().toUpperCase()
Original resourceName: UseHelper$Helper.groovy
ResourceName after handling dependency: Helper.groovy
Attempting to load resource: root/scripts/Helper.groovy
Attempting to fetch S3 object with key: testClient/root/scripts/Helper.groovy
Content loaded successfully for root/scripts/Helper.groovy
Content:
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
new Helper();
Edit: Adding ByteArrayURLConnection
class ByteArrayURLConnection extends URLConnection {
private final byte[] content
ByteArrayURLConnection(String spec, byte[] content) throws MalformedURLException {
super(new URL("file", "", -1, spec))
this.content = content
}
@Override
void connect() {}
@Override
InputStream getInputStream() {
return new ByteArrayInputStream(content)
}
}
seems there is a feature (bug?) in groovy - I see the resource files loaded multiple times during compilation process.
So, when you call ByteArrayURLConnection.constructor => super(new URL("file", "", -1, spec)) the groovy later is calling getURL() and trying to read content again but failing on a fake url - you can check it by overriding getURL() method.
I made your code working by using URL(URL context, String spec, URLStreamHandler handler) constructor and used URLStreamHandler to resolve the resource content. The idea is to provide a handler that knows how to open this url.
Here is a working groovy code that you can run in groovy console:
//i think this class could be a singleton and read resource from S3
class MyStreamHandler extends URLStreamHandler{
var src = [
'UseHelper': '''
def helper = new Helper()
helper.getMessage().toUpperCase()
''',
'Helper': '''
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
'''
]
protected URLConnection openConnection(URL u){
var path = u.getPath()
var text = src[path]
println "try get source ${u} => ${path} => ${text}"
if(!text)throw new IOException("not found: ${path}")
return new ByteArrayURLConnection(u, text.getBytes("UTF-8"))
}
}
class ByteArrayURLConnection extends URLConnection {
private final byte[] content
ByteArrayURLConnection(URL u, byte[] content) throws MalformedURLException {
super(u)
this.content = content
}
@Override
void connect() {}
@Override
InputStream getInputStream() {
return new ByteArrayInputStream(content)
}
}
class RC implements ResourceConnector{
@Override
URLConnection getResourceConnection(String resourceName) throws ResourceException {
var key = resourceName.replaceAll('.groovy$','')
try{
var u = new URL(new URL("file:"), key, new MyStreamHandler())
return u.openConnection()
}catch(e){
throw new ResourceException("not found: ${key}", e)
}
}
}
def engine = new GroovyScriptEngine(new RC())
def binding = new Binding()
def result = engine.run("UseHelper.groovy", binding)
println "result=$result"
output:
try get source file:UseHelper => UseHelper =>
def helper = new Helper()
helper.getMessage().toUpperCase()
try get source file:UseHelper$Helper => UseHelper$Helper => null
try get source file:groovy/lang/Script$Helper => groovy/lang/Script$Helper => null
try get source file:groovy/lang/GroovyObjectSupport$Helper => groovy/lang/GroovyObjectSupport$Helper => null
try get source file:groovy/lang/GroovyObject$Helper => groovy/lang/GroovyObject$Helper => null
try get source file:java/lang/Helper => java/lang/Helper => null
try get source file:java/util/Helper => java/util/Helper => null
try get source file:java/io/Helper => java/io/Helper => null
try get source file:java/net/Helper => java/net/Helper => null
try get source file:groovy/lang/Helper => groovy/lang/Helper => null
try get source file:groovy\lang\Helper => groovy\lang\Helper => null
try get source file:groovy/util/Helper => groovy/util/Helper => null
try get source file:groovy\util\Helper => groovy\util\Helper => null
try get source file:Helper => Helper =>
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
try get source file:Helper => Helper =>
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
try get source file:Helper => Helper =>
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
try get source file:UseHelper => UseHelper =>
def helper = new Helper()
helper.getMessage().toUpperCase()
try get source file:Helper => Helper =>
class Helper {
def getMessage() {
return "Hello from Helper class"
}
}
try get source file:UseHelperBeanInfo => UseHelperBeanInfo => null
try get source file:UseHelperCustomizer => UseHelperCustomizer => null
try get source file:HelperBeanInfo => HelperBeanInfo => null
try get source file:HelperCustomizer => HelperCustomizer => null
result=HELLO FROM HELPER CLASS