What is the best way to drag 3D point with mouse picking. Issue is not the picking but to dragging in 3D space.
There are two ways I am thinking, one is to get the View to World coordinates using gluUnProject and translate the 3D point. The problem in this case is, it only world on surfaces with Depth value (with glReadPixels), if mouse leaves the surface it gives the maximum or minimum depth values based on winZ component of gluUnProject. And it doesn't work in some cases.
The second way is to drag along XY, XZ, YZ plane using GL_MODELVIEW_MATRIX. But the problem in this case, is how would we know that we are on XY or XZ or YZ plane? How can we know that the front view of the trackball is in XY plane, and what if we want to drag on the side plane not the front plane?
So, is there any way that gives me the accurate 2D to 3D coordinate so that I can drag 3D point easily in every direction without considering the plane cases? There must be some ways, I have seen 3D softwares, they have perfect dragging feature.
I'm used to solving these user-interaction problems somewhat naively (perhaps not in a mathematically optimal way) but "well enough" considering that they're not very performance-critical (the user interaction parts, not necessarily the resulting modifications to the scene).
For unconstrained free dragging of an object, the method you described using unproject tends to work quite well to often give near pixel-perfect dragging with a slight tweak:
... instead of using glReadPixels to try to extract screen depth, you want a notion of a geometric object/mesh when the user is picking and/or has selected. Now just project the center point of that object to get your screen depth. Then you can drag around in screen X/Y, keeping the same Z you got from this projection, and unproject to get the resulting translation delta from previous center to new center to transform the object. This also makes it "feel" like you're dragging from the center of the object which tends to be quite intuitive.
For auto-constrained dragging, a quick way to detect that is to first grab a 'viewplane normal'. A quick way (might make mathematicians frown) using those projection/unprojection functions you're used to is to unproject two points at the center of the viewport in screenspace (one with a near z value and one with a far z value) and get a unit vector in between those two points. Now you can find the world axis closest to that normal using dot product. The other two world axises define the world plane we want to drag along.
Then it becomes a simple matter of using those handy unprojection functions again to get a ray along the mouse cursor. After that, you can do repeated ray/plane intersections as you're dragging the cursor around to compute a translation vector from the delta.
For more flexible constraints, a gizmo (aka manipulator, basically a 3D widget) can come in handy so that the user can indicate what kind of drag constraint he wants (planar, axis, unconstrained, etc) based on which parts of the gizmo he picks/drags. For axis constraints, ray/line or line/line intersection is handy.
As requested in the comments, to retrieve a ray from a viewport (C++-ish pseudocode):
// Get a ray from the current cursor position (screen_x and screen_y).
const float near = 0.0f;
const float far = 1.0f;
Vec3 ray_org = unproject(Vec3(screen_x, screen_y, near));
Vec3 ray_dir = unproject(Vec3(screen_x, screen_y, far));
ray_dir -= ray_org;
// Normalize ray_dir (rsqrt should handle zero cases to avoid divide by zero).
const float rlen = rsqrt(ray_dir[0]*ray_dir[0] +
ray_dir[1]*ray_dir[1] +
ray_dir[2]*ray_dir[2]);
ray_dir[0] *= rlen;
ray_dir[1] *= rlen;
ray_dir[2] *= rlen;
Then we do a ray/plane intersection with the ray obtained from the mouse cursor to figure out where the ray intersects the plane when the user begins dragging (the intersection will give us a 3D point). After that, it's just translating the object by the deltas between the points gathered from repeatedly doing this as the user drags the mouse around. The object should intuitively follow the mouse while being moved along a planar constraint.
Axis dragging is basically the same idea, but we turn the ray into a line and do a line/line intersection (mouse line against the line for the axis constraint, giving us a nearest point since the lines generally won't perfectly intersect), giving us back a 3D point from which we can use the deltas to translate the object along the constrained axis.
Note that there are tricky edge cases involved with axis/planar dragging constraints. For example, if a plane is perpendicular to the viewing plane (or close), it can shoot off the object into infinity. The same kind of case exists with axis dragging along a line that is perpendicular, like trying to drag along the Z axis from a front viewport (X/Y viewing plane). So it's worth detecting those cases where the line/plane is perpendicular (or close) and prevent dragging in such cases, but that can be done after you get the basic concept working.
Another trick worth noting to improve the way things "feel" for some cases is to hide the mouse cursor. For example, with axis constraints, the mouse cursor could end up becoming very distant from the axis itself, and it might look/feel weird. So I've seen a number of commercial packages simply hide the mouse cursor in this case to avoid revealing that discrepancy between the mouse and the gizmo/handle, and it tends to feel a bit more natural as a result. When the user releases the mouse button, the mouse cursor is moved to the visual center of the handle. Note that you shouldn't do this hidden-cursor dragging for tablets (they're a bit of an exception).
This picking/dragging/intersection stuff can be very difficult to debug, so it's worth tackling it in babysteps. Set small goals for yourself, like just clicking a mouse button in a viewport somewhere to create a ray. Then you can orbit around and make sure the ray was created in the right position. Next you can try a simple test to see if that ray intersects a plane in the world (say X/Y) plane, and create/visualize the intersection point between the ray and plane, and make sure that's correct. Take it in small, patient babysteps, pacing yourself, and you'll have smooth, confident progress. Try to do too much at once and you can have very discouraging jarring progress trying to figure out where you went wrong.