androidkotlinopengl-esopengl-es-2.0

How to detect click on objects in OpenGL ES 2.0


I need the program to detect when I click on the drawn object How can this be done in OpenGL ES? I get an array of coordinates and draw them on the field. I should be able to click on rendered points, but I don't know how to do it Render code below

My points array and buffer array

 var ppl: Array<FloatArray> = arrayOf(
        floatArrayOf(30f,1f,0f,0f,0f,1f),  ///x,y,z,isActive,isWork,signal
        floatArrayOf(-10f,10f,0f,1f,1f,1f),
        floatArrayOf(0f,0f,0f,1f,0f,0f),
        floatArrayOf(70f,50f,50f,1f,0f,1f)
    )

var mBufferLen = intArrayOf(0, 0, 0, 0, 0, 0, 0, 0, 0, 0) //0/Floor/Floor2/Point/Ball/Drop/Grid/Size/Search/Work

My onSurfaceCreated fun

    override fun onSurfaceCreated(gl1: GL10, pConfig: javax.microedition.khronos.egl.EGLConfig?) {
        val gl = gl1 as GL11 //we need 1.1 functionality
        //set background frame color
        //gl.glEnable(GL11.GL_LIGHTING)
        gl.glClearColor(0.01f, 0.5f, 0.3f, 1.0f) //dark-green
        BuildDrop(gl)
        BuildFloor2(gl)
        BuildFloor(gl)
        BuildGrid(gl)
        BuildSizeGrid(gl, spinnerIndicator)
        BuildDropT(gl)
        BuildWorkMark(gl)
    }

My BuildDrop fun

    fun BuildDrop(gl: GL11) {
        //every drop has the same coordinates
        //we glRotate and glTranslate when drawing
        val vtx = floatArrayOf(
            // X, Y, Z
            0f, 0f, 0f,
            -1f, -1f, 0f,
            1f, -1f, 0f,
            -1f, -1f, 0f,
            1f, -1f, 0f,
            0f, -2f, 0f,
        )
        StoreVertexData(gl, vtx, mDROP) //store in GPU buffer
    }

Store Vertex fun

    fun StoreVertexData(gl: GL11, pVertices: FloatArray, pObjectNum: Int) {
        val buffer: FloatBuffer = ByteBuffer.allocateDirect(pVertices.size * 4) //float is 4 bytes
            .order(ByteOrder.nativeOrder()) // use the device hardware's native byte order
            .asFloatBuffer() // create a floating point buffer from the ByteBuffer
            .put(pVertices) // add the coordinates to the FloatBuffer
        gl.glBindBuffer(GL11.GL_ARRAY_BUFFER, pObjectNum) //bind as current object
        buffer.position(0)
        //allocate memory and write buffer data
        gl.glBufferData(GL11.GL_ARRAY_BUFFER, buffer.capacity() * 4, buffer, GL11.GL_STATIC_DRAW)
        gl.glBindBuffer(GL11.GL_ARRAY_BUFFER, 0) //unbind from buffer
        mBufferLen[pObjectNum] = buffer.capacity() / 3 //store for drawing
    }

My fun for Drawing points

    private fun DrawDropMarks(gl: GL11, pDropCoords: Array<FloatArray>) {
        var pDropCoords = pDropCoords
        var pAngX = 0f
        var pAngY = 0f
        val TriCnt = pDropCoords.size //triangle count
        var isActive = 10f
        var isWork = 10f
        var signal = 10f
        val centerX = (display.width/2)
        val centerY = (display.height/2)
        for (ctr in 0 until TriCnt) {
            gl.glColor4f(1f, 0f, 0f, 1.0f) // mark default color
            gl.glPushMatrix() //translate\rotate only affects this single triangle
            gl.glTranslatef(centerX.toFloat(), centerY.toFloat(), 0f)
            //gl.glTranslatef(pDropCoords[ctr][0], pDropCoords[ctr][1], pDropCoords[ctr][2])
            if (MultiBillboard) //calc each triangle billboard angle separately
            {
                var hypLen: Float
                val distX = mCamXpos - pDropCoords[ctr][0]
                val distY = mCamYpos - pDropCoords[ctr][1]
                val distZ = mCamZpos - pDropCoords[ctr][2]
                isActive = pDropCoords[ctr][3]
                isWork = pDropCoords[ctr][4]
                signal = pDropCoords[ctr][5]
                //hypotenuse in 2D
                hypLen = sqrt((distX * distX + distZ * distZ).toDouble()).toFloat() //across floor
                pAngY = 270 - atan2(distZ.toDouble(), distX.toDouble()).toFloat() * mRad2Deg.toFloat()
                //3rd dimension
                pAngX = atan2(distY.toDouble(), hypLen.toDouble()).toFloat() * mRad2Deg.toFloat()
/*                Log.d("x",pAngX.toString())
                Log.d("y",pAngY.toString())
                Log.d("hyp",hypLen.toString())*/
            }
            gl.glRotatef(pAngY, 0f, 1f, 0f)
            gl.glRotatef(pAngX, 1f, 0f, 0f)
            mark = (sin((System.currentTimeMillis() * 2 * Math.PI / FLASH_DURATION) * 0.5) + 0.5)
            //size of triangles
            gl.glScalef(lengthRatio, lengthRatio, lengthRatio)
            if (signal == 0f){
                ppl = remove(ppl,ctr)
                pl += pDropCoords[ctr]
            }
            if ((isActive == 1f)){
                //initialize vertex Buffer for triangle
                gl.glEnableClientState(GL10.GL_VERTEX_ARRAY)
                gl.glBindBuffer(GL11.GL_ARRAY_BUFFER, mSEARCH)
                gl.glVertexPointer(3, GL11.GL_FLOAT, 0, 0)
                if (isWork == 0f) {
                    gl.glColor4f(0.3f, 1.0f, 0f, 1.0f)
                }
                else gl.glColor4f(1f, 1.0f, 0f, 1.0f)// active mark color
                //gl.glColor4f(0.3f, 1.0f, 0f, 1.0f) // active mark color
                gl.glDrawArrays(GL11.GL_LINES, 0, mBufferLen[mSEARCH])
            } //single drop
            if ((isWork == 1f) && (isActive == 1f)){
                //initialize vertex Buffer for triangle
                gl.glEnableClientState(GL10.GL_VERTEX_ARRAY)
                gl.glBindBuffer(GL11.GL_ARRAY_BUFFER, mWORK)
                gl.glVertexPointer(3, GL11.GL_FLOAT, 0, 0)
                gl.glColor4f(1f, 1.0f, 0f, 1.0f) // active mark color
                //gl.glColor4f(0.3f, 1.0f, 0f, 1.0f) // active mark color
                gl.glDrawArrays(GL11.GL_LINES, 0, mBufferLen[mWORK])
            } //single drop
            gl.glPopMatrix() //done with this triangle
        }
        gl.glBindBuffer(GL11.GL_ARRAY_BUFFER, 0) //unbind from buffer
    }

Im new in opengl I tried to detect click in onTouch event but coordinates in onTouchEvent and my coordinates are different. I also read about the ray-picking algorithm, but is there some simpler way to detect clicking on objects in opengl es?


Solution

  • My answer will be a bit broad. There are two general ways of detecting clicks.

    1. Drawing all objects to a separate framebuffer, each object in distinct colors and reading the pixel color from the buffer.

      Pros: fairly easy to implement
      Cons: reading from the framebuffer is slow.
      Read more about it here and maybe here

    2. Ray-picking (as you've said)

    There are pros and cons to both methods. In my projects I always went for 1. It involves some math, but that's what OpenGL is about. I can't describe the math here, but I might be able to give you some pointers.

    Also you might want to learn OpenGL ES 2.0 fairly soon. There is some benefit of learning the fixed function pipeline (it's a bit easier), but 2.0 has way more features and is more efficient.