Hi, through the Animation tutorial, I can change the vertice position in surface, but when i tried to translate five points, the show image is also three points, and the code is that:
```
int main(int argc, char **argv) {
// initialize Easy3D.
initialize();
//-------------------------------------------------------------
// Create the default Easy3D viewer.
// Note: a viewer must be created before creating any drawables.
Viewer viewer(EXAMPLE_TITLE);
//-------------------------------------------------------------
//// We use the points and indices of the "bunny".
//const std::vector<vec3> &points = resource::bunny_vertices;
//// Each consecutive 3 indices represent a triangle.
//const std::vector<unsigned int> &indices = resource::bunny_indices;
double joint_3d[5][3] = {
{2, -10, 1},
{-5, 2, 1},
{0, 0, 0},
{3, -3, 1},
{7, 2, -4}
};
std::vector<vec3> vec1;
vec1.reserve(5);
for (int i = 0; i < 5; ++i) {
vec1.emplace_back(joint_3d[i][0], joint_3d[i][1], joint_3d[i][2]);
}
const std::vector<vec3> &points = vec1;
const std::vector<unsigned int>& indices = { 0, 1, 2, 3, 4 };
//-------------------------------------------------------------
// Create a TrianglesDrawable to visualize the surface of the "bunny".
viewer.set_background_color(vec4(0.6, 0.8, 0.8, 0.1));
auto surface = new TrianglesDrawable("faces");
surface->update_vertex_buffer(points, true);
surface->update_element_buffer(indices);
surface->set_uniform_coloring(vec4(0.1, 0.4, 0.8, 1.0));
viewer.add_drawable(std::shared_ptr<TrianglesDrawable>(surface));
viewer.fit_screen();
viewer.set_animation(true);
// Define an animation function to specify how vertex positions are updated.
// In this trivial example, we stretch the model along the Z-axis.
viewer.animation_func_ = [&](easy3d::Viewer* v) -> bool {
(void)v;
// map the vertex buffer into the client's address space
void* pointer = VertexArrayObject::map_buffer(GL_ARRAY_BUFFER, surface->vertex_buffer(), GL_WRITE_ONLY);
vec3* vertices = reinterpret_cast<vec3*>(pointer);
if (!vertices)
return false;
static float total_scale = 1.0f;
float scale = 0.2f;
if (total_scale > 1.5f) {
scale = 1.0f / total_scale;
total_scale = 1.0f;
}
else
total_scale *= scale;
for (std::size_t i = 0; i < points.size(); ++i)
{
/*vertices[i].z *= scale;
vertices[i].x *= scale;
vertices[i].y *= scale;*/
//vertices[i].z += 0.5;
//vertices[i].x += 0.5;
vertices[i].y -= 0.02;
}
// unmap the vertex buffer
VertexArrayObject::unmap_buffer(GL_ARRAY_BUFFER, surface->vertex_buffer());
viewer.update();
return true;
};
// run the viewer
return viewer.run();
}
```
Do you know why happened this.
Because you're using a TrianglesDrawable, but the "const std::vector<unsigned int>& indices = { 0, 1, 2, 3, 4 }" contains vertex indices ONLY valid for 1 triangle.
For your skeleton, I think you should use the following two types of drawables (instead of TrianglesDrawable):
* LinesDrawable: draw the skeleton. It is actually a set of line segments. You should organize the index buffer such that each consecutive index pair denotes a line segment in the skeleton;
* PointsDrawable: draw the set of vertices (better show them as spheres).
If I want to draw hand mesh vertives, just like bunny in tutorial_311, which drawables should i use, and can you tell me how to do that. I'm so sorry for taking up your time.
You should use the following two types of drawables:
- **LinesDrawable**: draw the skeleton. It is actually a set of line segments. You should organize the index buffer such that each consecutive index pair denotes a line segment in the skeleton;
- **PointsDrawable**: draw the set of vertices (and show them as spheres).
Yeah, i realize your thoughts, and I tried to add animation into Tutorials_301_Drawables, the main purpose is changing the position of vertices, and the skeleton follow the changes, the code is below:
```
int main(int argc, char** argv) {
// initialize Easy3D.
initialize();
Viewer viewer(EXAMPLE_TITLE);
viewer.set_usage("");
//-------------------------------------------------------------
double joint_3d[3][3] = {
{0.0, 0.0, 0.0},
{0.5, 0.5, 0.5},
{-2, 0, 2}
};
std::vector<vec3> vec1;
vec1.reserve(21);
for (int i = 0; i < 3; ++i) {
vec1.emplace_back(joint_3d[i][0], joint_3d[i][1], joint_3d[i][2]);
}
const std::vector<vec3>& points = vec1;
// The coordinates of the vertices.
//const std::vector<vec3>& points = resource::bunny_vertices;
// The indices represent how the vertices are connected to form triangles. The "bunny" is a triangle mesh, and thus
// each consecutive three indices represent a triangle.
//const std::vector<unsigned int>& indices = resource::bunny_indices;
//const std::vector<unsigned int>& indices = { 1, 2, 3 };
//std::cout << "{ ";
//for (auto it = indices.cbegin(); it != indices.cend(); ++it) {
// std::cout << *it << " ";
//}
//std::cout << "}";
auto vertices = new PointsDrawable("vertices");
vertices->update_vertex_buffer(points);
vertices->set_uniform_coloring(vec4(0.0f, 1.0f, 0.0f, 1.0f)); // RBGA
vertices->set_impostor_type(PointsDrawable::SPHERE);
vertices->set_point_size(20);
viewer.add_drawable(std::shared_ptr<PointsDrawable>(vertices));
auto bbox_drawable = new LinesDrawable("bbox");
const std::vector<vec3> bbox_points = {
vec3(joint_3d[0][0], joint_3d[0][1], joint_3d[0][2]), vec3(joint_3d[1][0], joint_3d[1][1], joint_3d[1][2]),vec3(joint_3d[2][0], joint_3d[2][1], joint_3d[2][2])
};
const std::vector<unsigned int> bbox_indices = {
0, 1, 0, 2, 1, 2
};
bbox_drawable->update_vertex_buffer(bbox_points);
bbox_drawable->update_element_buffer(bbox_indices);
bbox_drawable->set_uniform_coloring(vec4(0.0f, 0.0f, 1.0f, 1.0f)); // r, g, b, a
bbox_drawable->set_line_width(5.0f);
viewer.add_drawable(std::shared_ptr<LinesDrawable>(bbox_drawable));
viewer.fit_screen();
viewer.set_animation(true);
viewer.animation_func_ = [&](easy3d::Viewer* v) -> bool {
(void)v;
// map the vertex buffer into the client's address space
void* pointer = VertexArrayObject::map_buffer(GL_ARRAY_BUFFER, vertices->vertex_buffer(), GL_WRITE_ONLY);
vec3* vertices = reinterpret_cast<vec3*>(pointer);
if (!vertices)
return false;
static float total_scale = 1.0f;
float scale = 0.2f;
if (total_scale > 1.5f) {
scale = 1.0f / total_scale;
total_scale = 1.0f;
}
else
total_scale *= scale;
for (std::size_t i = 0; i < points.size(); ++i)
{
/*vertices[i].z *= scale;
vertices[i].x *= scale;
vertices[i].y *= scale;*/
//vertices[i].z += 0.5;
//vertices[i].x += 0.5;
vertices[i].y -= 0.2;
}
// unmap the vertex buffer
VertexArrayObject::unmap_buffer(GL_ARRAY_BUFFER, vertices->vertex_buffer());
viewer.update();
return true;
};
// run the viewer
return viewer.run();
}
```
As you can see, in "VertexArrayObject::unmap_buffer(GL_ARRAY_BUFFER, vertices->vertex_buffer());" line, it indicates "easy3d::Vec<3Ui64,float>" have no member "vertex_buffer", can you give me some recommendation, thank you very much!
Sorry, I found it not affordable to have multiple iterations for teaching others to use Easy3D.
I assume the users are confident with C++ and also know the basics of OpenGL.
But if you want and okay to make your data public, I can create a demo for you (this would be easier for me). If you prefer this, please save each of your frames into a PLY file (if you already have data of the frames separately), and post the files here.
Yes, as you say, I still have no enough experience in OpenGL, Here is the joints data in 50 frames.
[https://drive.google.com/drive/folders/1LzwHP2Nsznv61j60Arr2FwWn-wnQtZuZ?dmr=1&ec=wgc-drive-globalnav-goto](url)
And the following is the connection relationship between joints which represent skeletons,
```
edge = [
(0, 1),
(1, 2),
(2, 3),
(3, 4),
(0, 5),
(5, 6),
(6, 7),
(7, 8),
(5, 9),
(9, 10),
(10, 11),
(11, 12),
(9, 13),
(13, 14),
(14, 15),
(15, 16),
(13, 17),
(17, 18),
(18, 19),
(19, 20),
(0, 17),
]
```
Thank you for me such more recommendation.
Below is the complete code:
- change `const std::string data_dir = "/home/liangliang/Downloads/hand/";` to point to your actual path. **Note**: You need to ensure consistent file names. For example, change "file1.ply" to "file01.ply" so that all file names are of the same length, so they can be unambiguously sorted. An alternative is to save all frames into a single file (but I leave this implementation to you).
- by default, it visualizes only the FIRST frame.
- you can change `viewer.set_animation(false);` to `viewer.set_animation(true);` to visualize the animation.
I didn't show the animation by default because I found your frames seem to be random, and they don't form a smooth sequence of animation.
```c++
#include <easy3d/viewer/viewer.h>
#include <easy3d/renderer/drawable_points.h>
#include <easy3d/renderer/drawable_lines.h>
#include "easy3d/renderer/camera.h"
#include <easy3d/util/initializer.h>
#include "easy3d/util/file_system.h"
#include <easy3d/fileio/ply_reader_writer.h>
using namespace easy3d;
int main(int argc, char **argv) {
// initialize Easy3D.
initialize();
//-------------------------------------------------------------
// Create the default Easy3D viewer.
// Note: a viewer must be created before creating any drawables.
Viewer viewer(EXAMPLE_TITLE);
//-------------------------------------------------------------
// The directory containing the frames of the hand skeleton.
// Note: each file contains the 21 vertices of the hand.
const std::string data_dir = "/home/liangliang/Downloads/hand/";
std::vector<std::string> files;
file_system::get_files(data_dir, files, false);
std::sort(files.begin(), files.end());
typedef std::vector<vec3> Frame;
std::vector<Frame> frames(files.size());
for (std::size_t i=0; i<files.size(); ++i) {
const std::string file = data_dir + files[i];
// We could use the general PointCloudIO to load each file as a point cloud.
// But for this special case, we use the PlyReader (to avoid loading each file into a point cloud model).
io::PlyReader reader;
std::vector<io::Element> elements;
reader.read(file, elements);
if (elements.size() == 1) // each file only contains vertices property.
frames[i] = elements[0].vec3_properties[0];
}
// To ensure all vertices are within the view frustum, compute the bounding box of all vertices across frames.
Box3 box;
for (const auto& frame : frames)
for (const auto& v : frame)
box.grow(v);
viewer.camera()->setSceneBoundingBox(box.min_point(), box.max_point());
// Each consecutive 2 indices represent a line segment.
const std::vector<unsigned int> indices = {
0, 1, 1, 2, 2, 3, 3, 4, 0, 5, 5, 6, 6, 7, 7, 8, 5, 9, 9, 10, 10, 11, 11,
12,9, 13, 13, 14, 14, 15, 15, 16, 13, 17, 17, 18, 18, 19, 19, 20, 0, 17
};
//-------------------------------------------------------------
// We create two drawable to render the hand:
// - a PointsDrawable for the vertices of the hand.
// - a LinesDrawable for the skeleton of the hand.
auto vertices = new PointsDrawable("vertices");
auto lines = new LinesDrawable("lines");
// Render vertices as spheres
vertices->set_impostor_type(PointsDrawable::SPHERE);
vertices->set_point_size(15.0f);
vertices->set_color(vec4(0.8f,0.3f, 0.3f, 1.0f));
// Render lines as cylinders
lines->set_impostor_type(LinesDrawable::CYLINDER);
lines->set_line_width(5.0f);
lines->set_color(vec4(0.3f,0.3f, 0.8f, 1.0f));
// Upload the vertice to GPU. The second argument is 'true' to allow dynamic update.
vertices->update_vertex_buffer(frames[0], true);
lines->update_vertex_buffer(frames[0], true);
// Upload the vertex indices of the lines to the GPU.
lines->update_element_buffer(indices);
// Add the drawables to the viewer
viewer.add_drawable(std::shared_ptr<PointsDrawable>(vertices));
viewer.add_drawable(std::shared_ptr<LinesDrawable>(lines));
// allow animation
viewer.set_animation(false);
// Define an animation function to specify how vertex positions are updated.
// In this trivial example, we stretch the model along the Z-axis.
viewer.animation_func_ = [&](easy3d::Viewer* v) -> bool {
(void)v;
static std::size_t index = 1;
const auto& vts = frames[index % frames.size()];
vertices->update_vertex_buffer(vts, true);
lines->update_vertex_buffer(vts, true);
++index;
viewer.update();
return true;
};
// run the viewer
return viewer.run();
}
```
Yes, according to the this demo, I got a lot of inspiration and try to render hand mesh based on the known hand vertices.
I want to ask one more question, due to hand detection is real-time, so i can't feed all the frame into the array "frames" in advance, and how can i transfer new value into the "vts", my initial idea is putting the model inference process into the animation function and update the value of "vts" constantly, do you think this idea is reasonable?
I think your data actually doesn't represent an animation.
Of course, you can still treat your data as if it were a smooth animation and use the existing code framework.
I don't know how frequent your data acquisition is. But given that your data is quite small (I mean, not many MB or GB), it makes sense to **transfer data to the GPU only when you receive the new data**. That means, you only need to call `update_vertex_buffer(...)` and perhaps also `update_element_buffer(...)` functions for the relevant drawables when new data is acquired.
Yes, the previous data i uploaded is from FreiHand dataset and it's not smooth. Today i try to render the whole hand motion through consecutive frames and attach the video, in hand mesh show, the hand will turns red from time to time, do you this reason, thank you!
https://drive.google.com/file/d/1ChrZQLqB_MNOaFVz8ioWdPrMVd4cTjy1/view?usp=drive_link
The color alters because the orientation of your mesh models is not consistent.
- yellow: face/vertex normals are pointing outside (this is expected);
- red: the normals are pointing inward.
This feature is enabled by default becasue in graphics research the orientation is quite important.
For your use case, you can simply disable it by calling `set_distinct_back_color(false)` of your TrianglesDrawable. Then both sides will be rendered with the same color.
BTW, as your sequence doesn't represent a smooth animation, you should avoid using the animation function (as it updates at about 30 fps and is for real animation). The best is to update the GPU buffers at a specified interval or only then the data of a new fram is received (and you need to implement this).
Today I tried to add the 3d modeling into model inference, and didn't use animation function in the process. But came across some errors, when the program go into viewer.run(), it would stop suddenly, the log is below:
```
================================================================= program started ...
V 04/07/2025 16:43:22.704 logging.cpp:201] executable path: /home/goertek/workspace/simpleHand/tech_show/spike_handpoints_detection/build
V 04/07/2025 16:43:22.704 logging.cpp:202] current working dir: /home/goertek/workspace/simpleHand/tech_show/spike_handpoints_detection/build
V 04/07/2025 16:43:22.704 resource.cpp:51] resource directory: /home/goertek/software/Easy3D-main/resources
V 04/07/2025 16:43:23.007 viewer.cpp:292] OpenGL vendor: NVIDIA Corporation
V 04/07/2025 16:43:23.007 viewer.cpp:293] OpenGL renderer: NVIDIA GeForce RTX 4090 D/PCIe/SSE2
V 04/07/2025 16:43:23.007 viewer.cpp:294] OpenGL version requested: 3.2
V 04/07/2025 16:43:23.007 viewer.cpp:295] OpenGL version received: 3.2.0 NVIDIA 570.133.20
V 04/07/2025 16:43:23.007 viewer.cpp:296] GLSL version received: 1.50 NVIDIA via Cg compiler
V 04/07/2025 16:43:23.007 viewer.cpp:310] samples received: 4 (4 requested, max support is 32)
V 04/07/2025 16:43:23.007 viewer.cpp:315] viewer size (width/height): 800/600
V 04/07/2025 16:43:23.007 viewer.cpp:321] DPI scaling: 1
V 04/07/2025 16:43:24.147 resource.cpp:61] resource directory: /home/goertek/software/Easy3D-main/resources
V 04/07/2025 16:43:24.148 text_renderer.cpp:756] loaded font 'en_Earth-Normal.ttf' in texture (512 x 512)
V 04/07/2025 16:43:24.148 text_renderer.cpp:756] loaded font 'en_Roboto-Medium.ttf' in texture (512 x 512)
F 04/07/2025 16:43:24.168 logging.cpp:105] Application has crashed due to [SIGSEGV] signal
Invalid access to memory
Program is trying to read an invalid (unallocated, deleted or corrupted) or inaccessible memory.
=================================================================================
Easy3D has encountered a fatal error and has to abort. The error has been recorded
in the log file [/home/goertek/workspace/simpleHand/tech_show/spike_handpoints_detection/build/handrects_detection_3d.log].
Please report this issue with the complete log, a description of how to reproduce
the issue, and possibly your data to Liangliang Nan ([email protected]).
=================================================================================
*** Check failure stack trace (most recent call first): ***
```
Do you know why happened this phenomenon, thank you!
The log doesn't show useful information.
You can run the program in the Debug mode, and that might be helpful for you to figure out the issue.
Or, come up with a minimum example that can replicate your issue, and send me the code so I can check it for you.
I am able to run your code because there are many undefined variables. Besides, I also don't have OpenCV installed.
When looking through your code, I found you put "viewer.run();" within a for loop. This might cause issues (but don't know if this is related to yours). The run() function should be called only once.
Today I put "viewer.run();" outside the for loop and modified some mistakes in the early code. Now it can show 3D hand structure normally but it would not update position of joint and only show the first frame, I put the simple code below that you can run. Due to I need to obtain image data constantly, so must use while loop and update vertex's value based on the detection result. The ply file you can use I uploaded before, it just justify whether it can move normally.
```
#include <easy3d/viewer/viewer.h>
#include <easy3d/renderer/drawable_points.h>
#include <easy3d/renderer/drawable_lines.h>
#include "easy3d/renderer/camera.h"
#include <easy3d/util/initializer.h>
#include "easy3d/util/file_system.h"
#include <easy3d/fileio/ply_reader_writer.h>
#include <easy3d/viewer/multi_viewer.h>
using namespace easy3d;
#define EXAMPLE_TITLE "Hand_detection"
int main(int argc, char **argv) {
// initialize Easy3D.
initialize();
//-------------------------------------------------------------
// Create the default Easy3D viewer.
// Note: a viewer must be created before creating any drawables.
// Viewer viewer(EXAMPLE_TITLE);
MultiViewer viewer(2, 2, EXAMPLE_TITLE);
//-------------------------------------------------------------
// The directory containing the frames of the hand skeleton.
// Note: each file contains the 21 vertices of the hand.
const std::string data_dir = "/home/goertek/workspace/simpleHand/ply/hand_joints/";
std::vector<std::string> files;
file_system::get_files(data_dir, files, false);
std::sort(files.begin(), files.end());
typedef std::vector<vec3> Frame;
std::vector<Frame> frames(files.size());
for (std::size_t i=0; i<files.size(); ++i) {
const std::string file = data_dir + files[i];
// We could use the general PointCloudIO to load each file as a point cloud.
// But for this special case, we use the PlyReader (to avoid loading each file into a point cloud model).
io::PlyReader reader;
std::vector<io::Element> elements;
reader.read(file, elements);
if (elements.size() == 1) // each file only contains vertices property.
frames[i] = elements[0].vec3_properties[0];
}
// To ensure all vertices are within the view frustum, compute the bounding box of all vertices across frames.
Box3 box;
for (const auto& frame : frames)
for (const auto& v : frame)
box.grow(v);
viewer.camera()->setSceneBoundingBox(box.min_point(), box.max_point());
// Each consecutive 2 indices represent a line segment.
const std::vector<unsigned int> indices = {
0, 1, 1, 2, 2, 3, 3, 4, 0, 5, 5, 6, 6, 7, 7, 8, 5, 9, 9, 10, 10, 11, 11,
12,9, 13, 13, 14, 14, 15, 15, 16, 13, 17, 17, 18, 18, 19, 19, 20, 0, 17
};
//-------------------------------------------------------------
// We create two drawable to render the hand:
// - a PointsDrawable for the vertices of the hand.
// - a LinesDrawable for the skeleton of the hand.
auto vertices = new PointsDrawable("vertices");
auto lines = new LinesDrawable("lines");
// Render vertices as spheres
vertices->set_impostor_type(PointsDrawable::SPHERE);
vertices->set_point_size(15.0f);
vertices->set_color(vec4(0.8f,0.3f, 0.3f, 1.0f));
// Render lines as cylinders
lines->set_impostor_type(LinesDrawable::CYLINDER);
lines->set_line_width(5.0f);
lines->set_color(vec4(0.3f,0.3f, 0.8f, 1.0f));
// Upload the vertice to GPU. The second argument is 'true' to allow dynamic update.
vertices->update_vertex_buffer(frames[0], true);
lines->update_vertex_buffer(frames[0], true);
// Upload the vertex indices of the lines to the GPU.
lines->update_element_buffer(indices);
// Add the drawables to the viewer
// viewer.add_drawable(std::shared_ptr<PointsDrawable>(vertices));
// viewer.add_drawable(std::shared_ptr<LinesDrawable>(lines));
viewer.assign(0, 0, vertices);
viewer.assign(0, 0, lines);
viewer.assign(0, 1, vertices);
viewer.assign(0, 1, lines);
viewer.run();
size_t index = 1;
while (index<50) {
const auto& vts = frames[index];
vertices->update_vertex_buffer(vts, true);
lines->update_vertex_buffer(vts, true);
++index;
viewer.update();
}
// allow animation
// viewer.set_animation(true);
// // Define an animation function to specify how vertex positions are updated.
// // In this trivial example, we stretch the model along the Z-axis.
// viewer.animation_func_ = [&](easy3d::Viewer* v) -> bool {
// (void)v;
// static std::size_t index = 1;
// const auto& vts = frames[index % frames.size()];
// vertices->update_vertex_buffer(vts, true);
// lines->update_vertex_buffer(vts, true);
// ++index;
// viewer.update();
// return true;
// };
// // run the viewer
// return viewer.run();
return 0;
}
```
You need a mechanism to trigger the "update".
There are multiple options, for example, using another thread or using a timer. I recommend using Easy3D's timer (see some [examples](https://github.com/LiangliangNan/Easy3D/blob/main/tests/test_timer.cpp). Basically, it allows you to run a function at a fixed interval.
In case you receive new data from your device at a non-uniform interval, you should perform the update every time new data is received.
Actually, the interval between adjacent frames in real time is not fixed, so I think using timer may have some problems in practice. Could you please create a simple demo based on the above code which show how to update its value or some relative examples, thanks!
Then you should check your sensor's documentation (or consult the relevant people) to see how the user can be informed when new data is generated. This is completely out of the scope of using Easy3D.
Yes, Can you tell what mechanism of its update, Why when I change vertices' value use viewer.update() function in while loop, it have no movement in rendering, thank you!
After executing "viewer.run()", the viewer will show, and the program will stay within the run() function. This means that the "while (index<50) ... " will never be reached when the viewer is running; it is executed only after you close the viewer.
The Easy3D viewer is based on GLFW. You can check the [GLFW documentation](https://www.glfw.org/docs/3.3/quick_guide.html#quick_window_close)to better understand how the viewer works.