androidopengl-esarcoresceneview

ARCore Android Studio- How to place 2d textiew on back of the 3d Model android and rotate?


I have a 3d model and a viewRenderable layout

This is how i loaded the model

model = ModelRenderable.builder()
            .setSource(
                requireContext(),
                RenderableSource.builder()
                    .setSource(
                        requireContext(),
                        Uri.parse(path),
                        RenderableSource.SourceType.GLB
                    )
                    .setScale(scale)
                    .setRecenterMode(RenderableSource.RecenterMode.ROOT)
                    .build()
            )
            .build()

After loading the model I am trying to add it in the sceneview

   model?.thenAccept { modelRenderable ->
        addModelToScene(arFragment, anchor, modelRenderable)
    }

Here I have two nodes transformableNode(3dBadge) and textWrittenOverBadgeNode I have added setParent for textWrittenOverBadgeNode as transformableNode

 private fun addModelToScene(
        arFragment: CustomARCameraFragment,
        anchor: Anchor?,
        modelRenderable: ModelRenderable?,
    ) {
        val anchorNode = anchor?.let { AnchorNode(it) }

    val transformableNode =
        TransformableNode(arFragment.transformationSystem)
    transformableNode.localPosition = Vector3(0f, 0.2f, -0.2f)
    transformableNode.renderable = modelRenderable
    transformableNode.setParent(anchorNode)

 val textWrittenOverBadgeNode = Node()

        ViewRenderable.builder().setView(requireContext(), R.layout.layout_ar_write_over_model)
            .build()
            .thenAccept {
                textWrittenOverBadgeNode.renderable = it
                it.renderPriority = 0
                textWrittenOverBadgeNode.worldRotation = transformableNode.worldRotation

                textWrittenOverBadgeNode.setParent(transformableNode)
            }

This is how my layout looks like

<LinearLayout
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:orientation="vertical">

        <com.google.android.material.textview.MaterialTextView
            android:id="@+id/tv_bullet_text"
            style="@style/text_24_gentona_bold_black_86"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:textColor="#fff"
            android:elevation="12dp"
            android:paddingStart="2dp"
            android:paddingEnd="2dp"
            android:shadowColor="#402702"
            android:shadowDx="2"
            android:shadowDy="2"
            android:shadowRadius="5"
            android:text="HELLO" />
    </LinearLayout>

Now I'm rotating the text and object

val animator = createAnimator()
        animator?.target = transformableNode
        animator?.start()

But both text and badge is not aligned properly

private fun createAnimator(): ObjectAnimator? {
        // Node's setLocalRotation method accepts Quaternions as parameters.
        // First, set up orientations that will animate a circle.
        val orientation1 = Quaternion.axisAngle(Vector3(0.0f, 1.0f, 0.0f), 0f)
        val orientation2 = Quaternion.axisAngle(Vector3(0.0f, 1.0f, 0.0f), 120f)
        val orientation3 = Quaternion.axisAngle(Vector3(0.0f, 1.0f, 0.0f), 240f)
        val orientation4 = Quaternion.axisAngle(Vector3(0.0f, 1.0f, 0.0f), 360f)
        val orbitAnimation = ObjectAnimator()
        orbitAnimation.setObjectValues(orientation1, orientation2, orientation3, orientation4)

        // Next, give it the localRotation property.
        orbitAnimation.setPropertyName("localRotation")

        // Use Sceneform's QuaternionEvaluator.
        orbitAnimation.setEvaluator(QuaternionEvaluator())

        //  Allow orbitAnimation to repeat forever
        orbitAnimation.repeatCount = ObjectAnimator.INFINITE
        orbitAnimation.repeatMode = ObjectAnimator.RESTART
        orbitAnimation.duration = 4500
        orbitAnimation.interpolator = LinearInterpolator()

        orbitAnimation.setAutoCancel(true)
        return orbitAnimation
    }

enter image description here


Solution

  • Make sure whatever 3d obj you're using arcore converts the object as single object you can get front, back, right, left position, you can try setting localPostion or worldPosition using viewRenderable, you can try with renderPriority too.

     val textWrittenOverBadgeNode = Node()
    
        ViewRenderable.builder().setView(requireContext(), R.layout.layout_ar_write_over_model)
            .build()
            .thenAccept {
                textWrittenOverBadgeNode.renderable = it
                it.renderPriority = 0
                textWrittenOverBadgeNode.localPosition.z = transformableNode.localPosition.z - 0.1f
                textWrittenOverBadgeNode.setParent(transformableNode)
            }
    

    Or you can try the below steps

    Yes, you can do this by applying texture to your model, if it seems what you want to do. You can make a opaque texture and apply your textview on top of that. add below code to addModelToScene

    texture?.thenAccept { texture ->
    
                MaterialFactory.makeOpaqueWithTexture(requireContext(), texture)
                    .thenAccept { material: Material? ->
                        // Apply the material to the model
    
                        // Set the texture as a diffuse map
                        material?.setFloat3(
                            MaterialFactory.MATERIAL_COLOR,
                            com.google.ar.sceneform.rendering.Color(0.219f, 0.172f, 0.52f)
                        )
                        material?.setTexture(MaterialFactory.MATERIAL_TEXTURE, texture)
                        transformableNode.renderable?.material = material
                    }
                    .exceptionally { throwable: Throwable ->
                        val builder = AlertDialog.Builder(requireContext())
                        builder.setTitle("Material Load Err!")
                        builder.setMessage(throwable.message).show()
                        return@exceptionally null
                    }
            }
    

    You can create a textview and convert this textview as bitmap and use this bitmap to render on top of your model.

    fun loadBitmapFromView(v: View): Bitmap? {
            val bitmap = Bitmap.createBitmap(v.width, v.height, Bitmap.Config.ARGB_8888)
            val c = Canvas(bitmap)
            v.draw(c)
            return bitmap
        }
    

    Use below code to save your the bitmap to file

    fun saveMediaToStorage(
            bitmap: Bitmap,
            filePath: File,
            extension: Bitmap.CompressFormat? = Bitmap.CompressFormat.JPEG,
            onSuccess: () -> Unit,
        ) {
            try {
                // Output stream
                val fos: OutputStream = FileOutputStream(filePath)
    
                fos.use {
                    // Finally writing the bitmap to the output stream that we opened
                    bitmap.compress(extension, 100, it)
                }
                onSuccess()
            } catch (e: IOException) {
            }
        }
    

    You can build a texture using the bitmap like this

    val texture = Texture.builder()
                            .setSource(
                                requireContext(),
                                Uri.parse(filePath.absolutePath)
                            )
                            .setUsage(Texture.Usage.DATA)
                            .setSampler(
                                Texture.Sampler.builder()
                                    .setMagFilter(Texture.Sampler.MagFilter.LINEAR)
                                    .setMinFilter(Texture.Sampler.MinFilter.LINEAR_MIPMAP_LINEAR)
                                    .build()
                            ).build()